# Multi-level Attention network using text, audio and video for Depression Prediction

arXiv: https://arxiv.org/abs/1909.01417 · HTML: https://mlsanity.com/paper/1909.01417/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Sep 2019.

## Built on by (most cited first, 19 of 19)

- [CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimation](https://mlsanity.com/paper/2207.14087/) (2022)
- [Multi-modal Depression Estimation based on Sub-attentional Fusion](https://mlsanity.com/paper/2207.06180/) (2022)
- [Reading Between the Frames: Multi-Modal Depression Detection in Videos from Non-Verbal Cues](https://mlsanity.com/paper/2401.02746/) (2024)
- [DEPAC: a Corpus for Depression and Anxiety Detection from Speech](https://mlsanity.com/paper/2306.12443/) (2023)
- [Prediction of Depression Severity Based on the Prosodic and Semantic Features with Bidirectional LSTM and Time Distributed CNN](https://mlsanity.com/paper/2202.12456/) (2022)
- [Automatic Depression Detection via Learning and Fusing Features from Visual Cues](https://mlsanity.com/paper/2203.00304/) (2022)
- [Depression Detection and Analysis using Large Language Models on Textual and Audio-Visual Modalities](https://mlsanity.com/paper/2407.06125/) (2024)
- [Speech based Depression Severity Level Classification Using a Multi-Stage Dilated CNN-LSTM Model](https://mlsanity.com/paper/2104.04195/) (2021)
- [Multimodal Depression Classification Using Articulatory Coordination Features And Hierarchical Attention Based Text Embeddings](https://mlsanity.com/paper/2202.06238/) (2022)
- [Enhancing Depression Detection via Question-wise Modality Fusion](https://mlsanity.com/paper/2503.20496/) (2025)
- [Predicting Mood Disorder Symptoms with Remotely Collected Videos Using an Interpretable Multimodal Dynamic Attention Fusion Network](https://mlsanity.com/paper/2109.03029/) (2021)
- [Neural Architecture Searching for Facial Attributes-based Depression Recognition](https://mlsanity.com/paper/2201.09799/) (2022)
- [MOGAM: A Multimodal Object-oriented Graph Attention Model for Depression Detection](https://mlsanity.com/paper/2403.15485/) (2024)
- [Automatic Depression Assessment using Machine Learning: A Comprehensive Survey](https://mlsanity.com/paper/2506.18915/) (2025)
- [Cost-effective Models for Detecting Depression from Speech](https://mlsanity.com/paper/2302.09214/) (2023)
- [Explainable Multimodal Depression Recognition in Clinical Interviews via PHQ-Aligned Symptom Summarization](https://mlsanity.com/paper/2501.16106/) (2025)
- [Probabilistic Textual Time Series Depression Detection](https://mlsanity.com/paper/2511.04476/) (2025)
- [A Multi-Probe Audit of Clinical-Interview Depression Detection Benchmarks](https://mlsanity.com/paper/2605.23977/) (2026)
- [Foundation Model-based Evaluation of Neuropsychiatric Disorders: A Lifespan-Inclusive, Multi-Modal, and Multi-Lingual Study](https://mlsanity.com/paper/2512.20948/) (2025)

## Abstract (arXiv)

Depression has been the leading cause of mental-health illness worldwide. Major depressive disorder (MDD), is a common mental health disorder that affects both psychologically as well as physically which could lead to loss of lives. Due to the lack of diagnostic tests and subjectivity involved in detecting depression, there is a growing interest in using behavioural cues to automate depression diagnosis and stage prediction. The absence of labelled behavioural datasets for such problems and the huge amount of variations possible in behaviour makes the problem more challenging. This paper presents a novel multi-level attention based network for multi-modal depression prediction that fuses features from audio, video and text modalities while learning the intra and inter modality relevance. The multi-level attention reinforces overall learning by selecting the most influential features within each modality for the decision making. We perform exhaustive experimentation to create different regression models for audio, video and text modalities. Several fusions models with different configurations are constructed to understand the impact of each feature and modality. We outperform the current baseline by 17.52% in terms of root mean squared error.