mlsanityData to 7 Oct 2026

DepMamba: Progressive Fusion Mamba for Multimodal Depression Detection

arXiv 2409.15936 ↗First posted 24 Sep 2024cs.CY cs.CV cs.HCcited by 14

On the map: State-space sequence models (S4, Mamba) → Speech and audio SSMs

Abstract (arXiv)

Depression is a common mental disorder that affects millions of people worldwide. Although promising, current multimodal methods hinge on aligned or aggregated multimodal fusion, suffering two significant limitations: (i) inefficient long-range temporal modeling, and (ii) sub-optimal multimodal fusion between intermodal fusion and intramodal processing. In this paper, we propose an audio-visual progressive fusion Mamba for multimodal depression detection, termed DepMamba. DepMamba features two core designs: hierarchical contextual modeling and progressive multimodal fusion. On the one hand, hierarchical modeling introduces convolution neural networks and Mamba to extract the local-to-global features within long-range sequences. On the other hand, the progressive fusion first presents a multimodal collaborative State Space Model (SSM) extracting intermodal and intramodal information for each modality, and then utilizes a multimodal enhanced SSM for modality cohesion. Extensive experimental results on two large-scale depression datasets demonstrate the superior performance of our DepMamba over existing state-of-the-art methods. Code is available at https://github.com/Jiaxin-Ye/DepMamba.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Speech and audio SSMs, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 19 of its 41 references that are in the map. First posted on arXiv 24 Sep 2024.

Builds on

19 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Audio Mamba: Bidirectional State Space Model for Audio Representation Learning5 Jun

Other references · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  5. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  6. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,051
  7. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 495
  8. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
  9. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 418
  10. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
  11. 2019EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognitioncited by 226
  12. 2019AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognitioncited by 87
6 more
  1. 2024Graph Mamba: Towards Learning on Graphs with State Space Modelscited by 84
  2. 2016AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and Challengecited by 68
  3. 2022Temporal Modeling Matters: A Novel Temporal Emotional Modeling Approach for Speech Emotion Recognitioncited by 19
  4. 2024U2UData: A Large-scale Cooperative Perception Dataset for Swarm UAVs Autonomous Flightcited by 15
  5. 2024LMVD: A Large-Scale Multimodal Vlog Dataset for Depression Detection in the Wildcited by 8
  6. 2023Emo-DNA: Emotion Decoupling and Alignment Learning for Cross-Corpus Speech Emotion Recognitioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 5.

14 papers.

Sort:

By area:

  1. 2025CardiacMamba: A Multimodal RGB-RF Fusion Framework with State Space Models for Remote Physiological Measurementcited by 8
  2. 2025MUG: Pseudo Labeling Augmented Audio-Visual Mamba Network for Audio-Visual Video Parsingcited by 3
  3. 2025LocoMamba: Vision-Driven Locomotion via End-to-End Deep Reinforcement Learning with Mambacited by 3
  4. 2025Towards Stable Cross-Domain Depression Recognition under Missing Modalitiescited by 3
  5. 2025MMFformer: Multimodal Fusion Transformer Network for Depression Detectioncited by 1
  6. 2025Personality-guided Public-Private Domain Disentangled Hypergraph-Former Network for Multimodal Depression Detectioncited by 1
  7. 2026CAF-Mamba: Mamba-Based Cross-Modal Adaptive Attention Fusion for Multimodal Depression Detectioncited by 1
  8. 2026Dep-LLM: Training-Free Depression Diagnosis via Evidence-Guided Structured Multi-factor with Reliable LLM Reasoningcited by 1
  9. 2025Multi-modal Collaborative Optimization and Expansion Network for Event-assisted Single-eye Expression Recognitioncited by 0
  10. 2025Vision-Proprioception Fusion with Mamba2 in End-to-End Reinforcement Learning for Motion Controlcited by 0
  11. 2025An overview of neural architectures for self-supervised audio representation learning from masked spectrogramscited by 0
  12. 2026CoSyncDiT: Cognitive Synchronous Diffusion Transformer for Movie Dubbingcited by 0
2 more
  1. 2026Uncovering Latent Depression Severity for Binary Depression Detection via Advantage-weighting Rankingcited by 0
  2. 2026Reliability-aware Cross-sample Enhancement for Robust Multimodal Sentiment Analysiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025It Hears, It Sees too: Multi-Modal LLM for Depression Detection By Integrating Visual Understanding into Audio Language Models
  2. 2025ClinicalFMamba: Advancing Clinical Assessment using Mamba-based Multimodal Neuroimaging Fusion
  3. 2026AlignMamba-2: Enhancing Multimodal Fusion and Sentiment Analysis with Modality-Aware Mamba
  4. 2025MLlm-DR: Towards Explainable Depression Recognition with MultiModal Large Language Models
  5. 2025Mixture of Experts for Recognizing Depression from Interview and Reading Tasks
  6. 2025Retrieval-Augmented Multimodal Depression Detection
  7. 2026RAMamba-Net: A Reliability-Aware and Mamba-Based Multimodal Fusion Network for Auditory Attention Detection
  8. 2026CAGMamba: Context-Aware Gated Cross-Modal Mamba Network for Multimodal Sentiment Analysis