# Self-supervised representations in speech-based depression detection

arXiv: https://arxiv.org/abs/2305.12263 · HTML: https://mlsanity.com/paper/2305.12263/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 20 May 2023.

## Builds on (most cited first, 14 of 14)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [Layer-wise Analysis of a Self-supervised Speech Representation Model](https://mlsanity.com/paper/2107.04734/) (2021)
- [BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition](https://mlsanity.com/paper/2109.13226/) (2021)
- [Speech Emotion Recognition using Self-Supervised Features](https://mlsanity.com/paper/2202.03896/) (2022)
- [Automatic Depression Detection: An Emotional Audio-Textual Corpus and a GRU/BiLSTM-based Model](https://mlsanity.com/paper/2202.08210/) (2022)
- [Topic Modeling Based Multi-modal Depression Detection](https://mlsanity.com/paper/1803.10384/) (2018)
- [Climate and Weather: Inspecting Depression Detection via Emotion Recognition](https://mlsanity.com/paper/2204.14099/) (2022)
- [A Step Towards Preserving Speakers' Identity While Detecting Depression Via Speaker Disentanglement](https://mlsanity.com/paper/2206.09530/) (2022)
- [Tandem Multitask Training of Speaker Diarisation and Speech Recognition for Meeting Transcription](https://mlsanity.com/paper/2207.03852/) (2022)

## Built on by (most cited first, 18 of 18)

- [When LLMs Meets Acoustic Landmarks: An Efficient Approach to Integrate Speech into Large Language Models for Depression Detection](https://mlsanity.com/paper/2402.13276/) (2024)
- [Affect Recognition in Conversations Using Large Language Models](https://mlsanity.com/paper/2309.12881/) (2023)
- [Crossmodal ASR Error Correction with Discrete Speech Units](https://mlsanity.com/paper/2405.16677/) (2024)
- [Transferring speech-generic and depression-specific knowledge for Alzheimer's disease detection](https://mlsanity.com/paper/2310.04358/) (2023)
- [Spontaneous Speech-Based Suicide Risk Detection Using Whisper and Large Language Models](https://mlsanity.com/paper/2406.03882/) (2024)
- [Parameter Efficient Finetuning for Speech Emotion Recognition and Domain Adaptation](https://mlsanity.com/paper/2402.11747/) (2024)
- [Self-Supervised Embeddings for Detecting Individual Symptoms of Depression](https://mlsanity.com/paper/2406.17229/) (2024)
- [Confidence Estimation for Automatic Detection of Depression and Alzheimer's Disease Based on Clinical Interviews](https://mlsanity.com/paper/2407.19984/) (2024)
- [Predicting Depression in Screening Interviews from Interactive Multi-Theme Collaboration](https://mlsanity.com/paper/2502.12204/) (2025)
- [Hierarchical Self-Supervised Representation Learning for Depression Detection from Speech](https://mlsanity.com/paper/2510.08593/) (2025)
- [It Hears, It Sees too: Multi-Modal LLM for Depression Detection By Integrating Visual Understanding into Audio Language Models](https://mlsanity.com/paper/2511.19877/) (2025)
- [Investigating Acoustic-Textual Emotional Inconsistency Information for Automatic Depression Detection](https://mlsanity.com/paper/2412.18614/) (2024)
- [Explainable Multimodal Depression Recognition in Clinical Interviews via PHQ-Aligned Symptom Summarization](https://mlsanity.com/paper/2501.16106/) (2025)
- [Clinically Inspired Symptom-Guided Depression Detection from Emotion-Aware Speech Representations](https://mlsanity.com/paper/2602.15578/) (2026)
- [Test-Time Training for Depression Detection](https://mlsanity.com/paper/2404.05071/) (2024)
- [DepFlow: Disentangled Speech Generation to Mitigate Semantic Bias in Depression Detection](https://mlsanity.com/paper/2601.00303/) (2026)
- [Semi-Supervised Diseased Detection from Speech Dialogues with Multi-Level Data Modeling](https://mlsanity.com/paper/2601.04744/) (2026)
- [Can Conversational Temporal Dynamics Improve Depression Detection in Dyads? A Preliminary Investigation in Multi-Modality Perspectives](https://mlsanity.com/paper/2607.03744/) (2026)

## Abstract (arXiv)

This paper proposes handling training data sparsity in speech-based automatic depression detection (SDD) using foundation models pre-trained with self-supervised learning (SSL). An analysis of SSL representations derived from different layers of pre-trained foundation models is first presented for SDD, which provides insight to suitable indicator for depression detection. Knowledge transfer is then performed from automatic speech recognition (ASR) and emotion recognition to SDD by fine-tuning the foundation models. Results show that the uses of oracle and ASR transcriptions yield similar SDD performance when the hidden representations of the ASR model is incorporated along with the ASR textual information. By integrating representations from multiple foundation models, state-of-the-art SDD results based on real ASR were achieved on the DAIC-WOZ dataset.