# MDD-Net: Multimodal Depression Detection through Mutual Transformer

arXiv: https://arxiv.org/abs/2508.08093 · HTML: https://mlsanity.com/paper/2508.08093/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 32 references that are in the map.
- First posted on arXiv 11 Aug 2025.

## Builds on (most cited first, 8 of 8)

- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Tensor Fusion Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/1707.07250/) (2017)
- [Attention Augmented Convolutional Networks](https://mlsanity.com/paper/1904.09925/) (2019)
- [Automatic Depression Detection: An Emotional Audio-Textual Corpus and a GRU/BiLSTM-based Model](https://mlsanity.com/paper/2202.08210/) (2022)
- [Vision Transformers with Hierarchical Attention](https://mlsanity.com/paper/2106.03180/) (2021)
- [Global Self-Attention Networks for Image Recognition](https://mlsanity.com/paper/2010.03019/) (2020)
- [Depression Diagnosis and Analysis via Multimodal Multi-order Factor Fusion](https://mlsanity.com/paper/2301.00254/) (2022)
- [Multimodal Depression Classification Using Articulatory Coordination Features And Hierarchical Attention Based Text Embeddings](https://mlsanity.com/paper/2202.06238/) (2022)

## Built on by (most cited first, 1 of 1)

- [Uncovering Latent Depression Severity for Binary Depression Detection via Advantage-weighting Ranking](https://mlsanity.com/paper/2607.05901/) (2026)

## Abstract (arXiv)

Depression is a major mental health condition that severely impacts the emotional and physical well-being of individuals. The simple nature of data collection from social media platforms has attracted significant interest in properly utilizing this information for mental health research. A Multimodal Depression Detection Network (MDD-Net), utilizing acoustic and visual data obtained from social media networks, is proposed in this work where mutual transformers are exploited to efficiently extract and fuse multimodal features for efficient depression detection. The MDD-Net consists of four core modules: an acoustic feature extraction module for retrieving relevant acoustic attributes, a visual feature extraction module for extracting significant high-level patterns, a mutual transformer for computing the correlations among the generated features and fusing these features from multiple modalities, and a detection layer for detecting depression using the fused feature representations. The extensive experiments are performed using the multimodal D-Vlog dataset, and the findings reveal that the developed multimodal depression detection network surpasses the state-of-the-art by up to 17.37% for F1-Score, demonstrating the greater performance of the proposed system. The source code is accessible at https://github.com/rezwanh001/Multimodal-Depression-Detection.