Personality-guided Public-Private Domain Disentangled Hypergraph-Former Network for Multimodal Depression Detection

AI summary

Overview

• Research area: Multimodal machine learning for mental health, specifically automated depression detection from audio and video, drawing on graph/hypergraph neural networks, transformer attention, domain disentanglement, and large language models for personalized representation.
• Technical level: Advanced. The paper assumes familiarity with hypergraph convolution, adversarial (GAN-style) training, Hilbert-Schmidt Independence Criterion, contrastive learning, and multimodal feature fusion.
• Scope: The paper proposes P3HF, a personality-guided, public-private domain disentangled Hypergraph-Former network, and evaluates it on the MPDD-Young dataset for binary and ternary depression classification.

What This Paper Is About

Existing automated depression detection systems tend to model everyone the same way, ignore the order in which symptoms unfold over time, and treat each recording session as an isolated event. The authors argue that depression shows up differently in different people and in different behavioral contexts (in their data, three tasks: a self-introduction and two text-reading tasks), so a good model must personalize its representations, model high-order temporal cross-modal structure, and separate what is shared across events from what is specific to one event. Their goal is a single architecture that does all three and generalizes better to unseen individuals and events.

Key Contributions

1. Personality-guided feature regulation. Discrete individual attributes (gender, age, hometown, Big Five personality scores) are turned into descriptive text by an LLM and used to gate audio and visual features, so the same audiovisual signal is modulated differently for different people.
2. Hypergraph-Former architecture. Sinusoidal positional encoding and multi-head self-attention are added to a hypergraph neural network, giving it the temporal-order awareness that standard unordered hyperedges lack while retaining high-order intra-modal and inter-modal relationship modeling.
3. Event-level public-private domain disentanglement. A shared public encoder is trained adversarially against an event discriminator (to extract event-invariant features), while independent per-event private encoders are pushed apart using HSIC, improving generalization across behavioral contexts.
4. State-of-the-art results and analysis. Experiments on MPDD-Young plus component ablations, architecture comparisons, hyperparameter sensitivity, and t-SNE visualization of the disentanglement. Code is released at https://github.com/hacilab/P3HF.

Main Findings

• Overall performance on MPDD-Young: P3HF reaches 82.17% accuracy / 81.39% weighted F1 on binary classification and 76.29% / 74.61% on ternary classification (normal, mildly depressed, severely depressed). The abstract summarizes this as roughly 10% improvement in accuracy and weighted F1 over existing methods.
• Margin over the strongest baseline: DepMamba (2025) scores 72.56% / 71.44% binary and 67.85% / 66.23% ternary, so P3HF improves by 9.61% / 9.95% binary and 8.44% / 8.38% ternary.
• Other baselines: Unimodal NUSD (63.01% / 60.64% binary; 57.19% / 55.44% ternary) and STA-DRN (64.14% / 62.23%; 58.93% / 57.34%); multimodal Baseline 2025 (63.64% / 59.96%; 49.66% / 51.86%), Gated LSTM (64.48% / 62.17%; 52.51% / 50.32%), TBN (66.21% / 64.77%; 61.76% / 60.23%), IA fusion (68.41% / 67.23%; 62.87% / 61.39%), DEP-Former (67.85% / 66.23%; 63.43% / 61.75%), MGLRA (70.37% / 68.93%; 61.35% / 59.78%).
• Modality ablation: Removing visual features drops binary accuracy by 4.65% and ternary by 3.35%; removing audio drops binary by 5.28% and ternary by 5.44%, suggesting audio carries more discriminative temporal patterns in this setup.
• Domain disentanglement matters most: Removing domain disentanglement entirely gives 71.84% / 70.17% binary and 66.53% / 65.72% ternary. Removing the public-domain encoder costs 6.83% / 5.99%, while removing the private-domain encoders costs only 4.02% / 2.28%, which the authors read as evidence that shared, event-invariant information dominates.
• Personality guidance helps: Removing personal information drops performance by 5.49% / 4.74%; replacing the LLM-generated description with a numeric embedding gives 80.61% / 78.77% binary and 75.32% / 73.34% ternary, both below the full model.
• Architecture comparison: Cross-attention is weakest (75.82% / 69.15% binary/ternary accuracy); directed GCN (77.55% / 72.41%) is below undirected GCN (79.33% / 73.57%), which is below undirected GAT (80.07% / 74.23%); a plain hypergraph reaches 79.68% / 73.86%, and Hypergraph-Former surpasses it by 2.49% / 2.43%.
• Hyperparameters: Window size shows an inverted-U relationship with an optimum at 11; attention heads peak at 4. The authors note that the optimal discriminator accuracy sits around 1/3, which they treat as the signature of successful public-domain learning.
• Disentanglement visualization: t-SNE shows chaotic mixing across events with no disentanglement (β = γ = 0), incomplete separation with only one loss active, and clear separation of unified public features versus distinct private features at β = γ = 0.1.

Methodology in Plain English

Each subject provides three events of audio and video, split into frames. Faces are detected with Haar cascades and turned into 2048-dimensional features using pretrained ResNet-50; audio is turned into 1024-dimensional features using a Chinese fine-tuned Wav2Vec2. The demographic and Big Five personality scores are written into a prompt for GPT-4 (temperature set to 0 for reproducibility), and the resulting description of the person is encoded by BERT into 768-dimensional features. Events of different lengths are aligned by repetition-based padding.

Those personality features are pushed through a bidirectional LSTM and turned into a gate — a set of weights that decides how much each audiovisual feature should be amplified for this particular person. The gated features then receive positional encodings so their time order is preserved, and are placed into a hypergraph built per event: nodes are all audio and visual frames (2T_k nodes), and hyperedges are created inside sliding windows to connect same-modality nodes (intra-modal) and all nodes of one modality to all nodes of the other (inter-modal), yielding (T_k − w + 1) × (2 + 2w) hyperedges. Hypergraph convolution aggregates information, then multi-head self-attention captures global dependencies, and the audio and visual streams are concatenated.

The concatenated event representation is split into two streams. A shared public encoder tries to produce event-invariant features, trained adversarially against a discriminator that tries to guess which event a feature came from. Separate private encoders per event try to capture event-specific features, and HSIC is minimized between private representations of different events to keep them independent. The averaged public representation plus all private representations are concatenated and projected to three depression classes. Training minimizes a weighted sum of the NLL depression loss, the adversarial loss, and the HSIC loss, with α + β + γ = 1, using alternating updates between the discriminator and the main model. Experiments used a Windows 10 machine with an NVIDIA RTX 4090, PyTorch 1.13.1 and PyG 2.6.1, batch size 20, up to 300 epochs, cosine-annealed learning rate from 1e-4 to 1e-5 tuned with Optuna, Adam with weight decay 5e-4, early stopping, and 10 random seeds with a one-way ANOVA (p < 0.05).

Why This Matters

• Research impact: The paper pushes depression detection from population-level, single-event modeling toward individual-aware, multi-event modeling. It also makes a specific architectural claim: unordered hyperedges lose temporal information, and bolting positional encoding plus attention onto a hypergraph recovers it. Event-level disentanglement in depression detection is presented as new.
• Clinical screening tools: automated pre-screening from short video and audio recordings, particularly where mental health specialists are scarce.
• Telehealth and remote monitoring: the multi-event setting maps naturally to online sessions where a patient completes several structured tasks, and where the event-invariant component should transfer to new activities.
• Personalized digital mental health: the personality-gating mechanism could be reused wherever the same sensor data should be interpreted differently for different users.
• Industry relevance: companies building affect-aware or mental-wellness products, together with the broader multimodal fusion community, can reuse the Hypergraph-Former and public-private disentanglement modules beyond depression. The paper also notes the approach may generalize to anxiety disorders and bipolar affective disorder, though this is stated as a promise rather than demonstrated.

Future Directions

• Validation beyond one dataset: all results come from MPDD-Young; the authors report no cross-dataset or cross-population evaluation, and no participant count or dataset size is reported in the paper content.
• Extension to other conditions: the conclusion explicitly proposes applying the framework to anxiety disorders and bipolar affective disorder and to "more general model applications in related directions."
• More individual information sources: the current personality prompt uses gender, age, hometown, and Big Five scores; the paper does not explore richer or longitudinal personal histories, nor the sensitivity of results to the LLM-generated descriptions themselves.
• Balancing the disentanglement losses: the ablation shows removing the public encoder hurts more than removing the private encoders; how far the public/private split should be pushed, and whether β = γ = 0.1 transfers to other datasets or numbers of events (the study uses K = 3), remain open.

Target Audience

Researchers and graduate students in multimodal affective computing, computational psychiatry, and graph/transformer architecture design; practitioners building automated mental health screening systems who need a template for combining LLM-derived personalization with structured spatio-temporal reasoning; and readers interested in domain disentanglement applied to multi-event behavioral data rather than to conventional visual or text domains.

Authors’ abstract

Depression represents a global mental health challenge requiring efficient and reliable automated detection methods. Current Transformer- or Graph Neural Networks (GNNs)-based multimodal depression detection methods face significant challenges in modeling individual differences and cross-modal temporal dependencies across diverse behavioral contexts. Therefore, we propose P$^3$HF (Personality-guided Public-Private Domain Disentangled Hypergraph-Former Network) with three key innovations: (1) personality-guided representation learning using LLMs to transform discrete individual features into contextual descriptions for personalized encoding; (2) Hypergraph-Former architecture modeling high-order cross-modal temporal relationships; (3) event-level domain disentanglement with contrastive learning for improved generalization across behavioral contexts. Experiments on MPDD-Young dataset show P$^3$HF achieves around 10\% improvement on accuracy and weighted F1 for binary and ternary depression classification task over existing methods. Extensive ablation studies validate the independent contribution of each architectural component, confirming that personality-guided representation learning and high-order hypergraph reasoning are both essential for generating robust, individual-aware depression-related representations. The code is released at https://github.com/hacilab/P3HF.