Explainable Multimodal Depression Recognition in Clinical Interviews via PHQ-Aligned Symptom Summarization
Not on the map. Why?
Abstract (arXiv)
Recent advances in multimodal depression recognition for clinical interviews (MDRC) have demonstrated the potential of AI systems by integrating textual, acoustic, and facial cues. However, existing methods pay limited attention to interpretability, thereby constraining reproducibility and clinician review. To address this, we introduce Explain-MDRC, an explainable MDRC framework that mirrors clinical workflows by generating structured symptom summaries from text and integrating them with nonverbal cues for recognition. Specifically, we construct Explain-DAIC, a dataset based on DAIC-WOZ and enriched with PHQ-8-aligned summary annotations, providing a foundation for developing models with built-in interpretability. We further propose PhqCML, a model that combines PHQ-8-aligned symptom summarization with PHQ-aware contrastive learning and summary-informed multimodal fusion. Automated metrics and expert evaluations show that Explain-MDRC improves recognition performance and provides more interpretable, clinician-readable intermediate evidence, suggesting a promising direction for transparent AI-assisted depression recognition research.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 36 of its 83 references that are in the map. First posted on arXiv 27 Jan 2025.
Builds on
36 of 83 references.
References · 36
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2020Supervised Contrastive Learningcited by 2,829
- 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 645
- 2022Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Frameworkcited by 379
- 2020MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysiscited by 265
- 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 246
- 2024Capabilities of Gemini Models in Medicinecited by 237
- 2023BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Taskscited by 201
- 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 85
- 2023Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicinecited by 83
- 2016AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and Challengecited by 69
- 2023Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Datacited by 66
24 more
- 2020Generating SOAP Notes from Doctor-Patient Conversations Using Modular Summarization Techniquescited by 64
- 2023MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Modelscited by 63
- 2023CLIPSyntel: CLIP and LLM Synergy for Multimodal Question Summarization in Healthcarecited by 43
- 2020MODMA dataset: a Multi-modal Open Dataset for Mental-disorder Analysiscited by 38
- 2022Automatic Depression Detection: An Emotional Audio-Textual Corpus and a GRU/BiLSTM-based Modelcited by 37
- 2022D4: a Chinese Dialogue Dataset for Depression-Diagnosis-Oriented Chatcited by 37
- 2018Topic Modeling Based Multi-modal Depression Detectioncited by 29
- 2024PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agentscited by 28
- 2022Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnairescited by 27
- 2024MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognitioncited by 27
- 2020Identifying Depressive Symptoms from Tweets: Figurative Language Enabled Multitask Learning Frameworkcited by 25
- 2022Learning to Automate Follow-up Question Generation using Process Knowledge for Depression Triage on Reddit Postscited by 23
- 2022Symptom Identification for Interpretable Detection of Multiple Mental Disorderscited by 20
- 2019Multi-level Attention network using text, audio and video for Depression Predictioncited by 19
- 2023Self-supervised representations in speech-based depression detectioncited by 18
- 2024When LLMs Meets Acoustic Landmarks: An Efficient Approach to Integrate Speech into Large Language Models for Depression Detectioncited by 17
- 2024HiQuE: Hierarchical Question Embedding Network for Multimodal Depression Detectioncited by 12
- 2024Towards Context-Aware Emotion Recognition Debiasing from a Causal Demystification Perspective via De-confounded Trainingcited by 11
- 2024LMVD: A Large-Scale Multimodal Vlog Dataset for Depression Detection in the Wildcited by 8
- 2024Tracing Intricate Cues in Dialogue: Joint Graph Structure and Sentiment Dynamics for Multimodal Emotion Recognitioncited by 8
- 2022Prediction of Depression Severity Based on the Prosodic and Semantic Features with Bidirectional LSTM and Time Distributed CNNcited by 7
- 2023Node-weighted Graph Convolutional Network for Depression Detection in Transcribed Clinical Interviewscited by 7
- 2022PTSD in the Wild: A Video Database for Studying Post-Traumatic Stress Disorder Recognition in Unconstrained Environmentscited by 6
- 2025LLAMADRS: Evaluating Open-Source LLMs on Real Clinical Interviews--To Reason or Not to Reason?cited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.