# A knowledge-driven vowel-based approach of depression classification from speech using data augmentation

arXiv: https://arxiv.org/abs/2210.15261 · HTML: https://mlsanity.com/paper/2210.15261/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 33 references that are in the map.
- First posted on arXiv 27 Oct 2022.

## Builds on (most cited first, 9 of 9)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and Challenge](https://mlsanity.com/paper/1605.01600/) (2016)
- [KeepAugment: A Simple Information-Preserving Data Augmentation Approach](https://mlsanity.com/paper/2011.11778/) (2020)
- [Gender Bias in Depression Detection Using Audio Features](https://mlsanity.com/paper/2010.15120/) (2020)
- [SpeechFormer: A Hierarchical Efficient Framework Incorporating the Characteristics of Speech](https://mlsanity.com/paper/2203.03812/) (2022)
- [Adversarial Data Augmentation for Disordered Speech Recognition](https://mlsanity.com/paper/2108.00899/) (2021)
- [FrAUG: A Frame Rate Based Data Augmentation Method for Depression Detection from Speech Signals](https://mlsanity.com/paper/2202.05912/) (2022)
- [Toward Knowledge-Driven Speech-Based Models of Depression: Leveraging Spectrotemporal Variations in Speech Vowels](https://mlsanity.com/paper/2210.02527/) (2022)

## Built on by (most cited first, 5 of 5)

- [Attention-Based Acoustic Feature Fusion Network for Depression Detection](https://mlsanity.com/paper/2308.12478/) (2023)
- [Robust and Explainable Depression Identification from Speech Using Vowel-Based Ensemble Learning Approaches](https://mlsanity.com/paper/2410.18298/) (2024)
- [Investigating Acoustic-Textual Emotional Inconsistency Information for Automatic Depression Detection](https://mlsanity.com/paper/2412.18614/) (2024)
- [A Novel Data Augmentation Approach for Automatic Speaking Assessment on Opinion Expressions](https://mlsanity.com/paper/2506.04077/) (2025)
- [DepFlow: Disentangled Speech Generation to Mitigate Semantic Bias in Depression Detection](https://mlsanity.com/paper/2601.00303/) (2026)

## Abstract (arXiv)

We propose a novel explainable machine learning (ML) model that identifies depression from speech, by modeling the temporal dependencies across utterances and utilizing the spectrotemporal information at the vowel level. Our method first models the variable-length utterances at the local-level into a fixed-size vowel-based embedding using a convolutional neural network with a spatial pyramid pooling layer ("vowel CNN"). Following that, the depression is classified at the global-level from a group of vowel CNN embeddings that serve as the input of another 1D CNN ("depression CNN"). Different data augmentation methods are designed for both the training of vowel CNN and depression CNN. We investigate the performance of the proposed system at various temporal granularities when modeling short, medium, and long analysis windows, corresponding to 10, 21, and 42 utterances, respectively. The proposed method reaches comparable performance with previous state-of-the-art approaches and depicts explainable properties with respect to the depression outcome. The findings from this work may benefit clinicians by providing additional intuitions during joint human-ML decision-making tasks.