# Multi-modal Depression Estimation based on Sub-attentional Fusion

arXiv: https://arxiv.org/abs/2207.06180 · HTML: https://mlsanity.com/paper/2207.06180/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 62 references that are in the map.
- First posted on arXiv 13 Jul 2022.

## Builds on (most cited first, 17 of 17)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations](https://mlsanity.com/paper/2106.01548/) (2021)
- [Attentional Feature Fusion](https://mlsanity.com/paper/2009.14082/) (2020)
- [Uncertainty-aware Score Distribution Learning for Action Quality Assessment](https://mlsanity.com/paper/2006.07665/) (2020)
- [AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and Challenge](https://mlsanity.com/paper/1605.01600/) (2016)
- [Topic Modeling Based Multi-modal Depression Detection](https://mlsanity.com/paper/1803.10384/) (2018)
- [Deep Learning for Depression Recognition with Audiovisual Cues: A Review](https://mlsanity.com/paper/2106.00610/) (2021)
- [Measuring Depression Symptom Severity from Spoken Language and 3D Facial Expressions](https://mlsanity.com/paper/1811.08592/) (2018)
- [Gender Bias in Depression Detection Using Audio Features](https://mlsanity.com/paper/2010.15120/) (2020)
- [Multi-level Attention network using text, audio and video for Depression Prediction](https://mlsanity.com/paper/1909.01417/) (2019)
- [Depression Severity Estimation from Multiple Modalities](https://mlsanity.com/paper/1711.06095/) (2017)
- [Automatic Depression Detection via Learning and Fusing Features from Visual Cues](https://mlsanity.com/paper/2203.00304/) (2022)
- [Interpreting Depression From Question-wise Long-term Video Recording of SDS Evaluation](https://mlsanity.com/paper/2106.13393/) (2021)
- [Depression Scale Recognition from Audio, Visual and Text Analysis](https://mlsanity.com/paper/1709.05865/) (2017)
- [Detecting Levels of Depression in Text Based on Metrics](https://mlsanity.com/paper/1807.03397/) (2018)

## Built on by (most cited first, 15 of 15)

- [U-Fair: Uncertainty-based Multimodal Multitask Learning for Fairer Depression Detection](https://mlsanity.com/paper/2501.09687/) (2025)
- [Reading Between the Frames: Multi-Modal Depression Detection in Videos from Non-Verbal Cues](https://mlsanity.com/paper/2401.02746/) (2024)
- [Multimodal Gender Fairness in Depression Prediction: Insights on Data from the USA & China](https://mlsanity.com/paper/2408.04026/) (2024)
- [Towards Activated Muscle Group Estimation in the Wild](https://mlsanity.com/paper/2303.00952/) (2023)
- [Integrating Large Language Models into a Tri-Modal Architecture for Automated Depression Classification on the DAIC-WOZ](https://mlsanity.com/paper/2407.19340/) (2024)
- [Leveraging Audio and Text Modalities in Mental Health: A Study of LLMs Performance](https://mlsanity.com/paper/2412.10417/) (2024)
- [MLlm-DR: Towards Explainable Depression Recognition with MultiModal Large Language Models](https://mlsanity.com/paper/2507.05591/) (2025)
- [Attention-Based Acoustic Feature Fusion Network for Depression Detection](https://mlsanity.com/paper/2308.12478/) (2023)
- [Predicting Depression in Screening Interviews from Interactive Multi-Theme Collaboration](https://mlsanity.com/paper/2502.12204/) (2025)
- [Automatic Depression Assessment using Machine Learning: A Comprehensive Survey](https://mlsanity.com/paper/2506.18915/) (2025)
- [It Hears, It Sees too: Multi-Modal LLM for Depression Detection By Integrating Visual Understanding into Audio Language Models](https://mlsanity.com/paper/2511.19877/) (2025)
- [Evaluating Lexicon Incorporation for Depression Symptom Estimation](https://mlsanity.com/paper/2404.19359/) (2024)
- [Tri-Modal Severity Fused Diagnosis across Depression and Post-traumatic Stress Disorders](https://mlsanity.com/paper/2510.20239/) (2025)
- [IDRL: An Individual-Aware Multimodal Depression-Related Representation Learning Framework for Depression Diagnosis](https://mlsanity.com/paper/2603.11644/) (2026)
- [MA-DLE: Speech-based Automatic Depression Level Estimation via Memory Augmentation](https://mlsanity.com/paper/2606.11197/) (2026)

## Abstract (arXiv)

Failure to timely diagnose and effectively treat depression leads to over 280 million people suffering from this psychological disorder worldwide. The information cues of depression can be harvested from diverse heterogeneous resources, e.g., audio, visual, and textual data, raising demand for new effective multi-modal fusion approaches for automatic estimation. In this work, we tackle the task of automatically identifying depression from multi-modal data and introduce a sub-attention mechanism for linking heterogeneous information while leveraging Convolutional Bidirectional LSTM as our backbone. To validate this idea, we conduct extensive experiments on the public DAIC-WOZ benchmark for depression assessment featuring different evaluation modes and taking gender-specific biases into account. The proposed model yields effective results with 0.89 precision and 0.70 F1-score in detecting major depression and 4.92 MAE in estimating the severity. Our attention-based fusion module consistently outperforms conventional late fusion approaches and achieves competitive performance compared to the previously published depression estimation frameworks, while learning to diagnose the disorder end-to-end and relying on far fewer preprocessing steps.