# CO-VADA: A Confidence-Oriented Voice Augmentation Debiasing Approach for Fair Speech Emotion Recognition

arXiv: https://arxiv.org/abs/2506.06071 · HTML: https://mlsanity.com/paper/2506.06071/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 65 references that are in the map.
- First posted on arXiv 6 Jun 2025.

## Builds on (most cited first, 30 of 39)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [End-to-End Bias Mitigation by Modelling Biases in Corpora](https://mlsanity.com/paper/1909.06321/) (2019)
- [Learning Debiased Representation via Disentangled Feature Augmentation](https://mlsanity.com/paper/2107.01372/) (2021)
- [FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversion](https://mlsanity.com/paper/2210.15418/) (2022)
- [BiaSwap: Removing dataset bias with bias-tailored swapping augmentation](https://mlsanity.com/paper/2108.10008/) (2021)
- [Voice Conversion With Just Nearest Neighbors](https://mlsanity.com/paper/2305.18975/) (2023)
- [Domain Adversarial Training for Accented Speech Recognition](https://mlsanity.com/paper/1806.02786/) (2018)
- [Balancing out Bias: Achieving Fairness Through Balanced Training](https://mlsanity.com/paper/2109.08253/) (2021)
- [Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-shot Speaker Adaptation](https://mlsanity.com/paper/2311.04693/) (2023)
- [Toward Fairness in Speech Recognition: Discovery and mitigation of performance disparities](https://mlsanity.com/paper/2207.11345/) (2022)
- [Kid-Whisper: Towards Bridging the Performance Gap in Automatic Speech Recognition for Children VS. Adults](https://mlsanity.com/paper/2309.07927/) (2023)
- [BLIND: Bias Removal With No Demographics](https://mlsanity.com/paper/2212.10563/) (2022)
- [Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Models](https://mlsanity.com/paper/2407.06957/) (2024)
- [Don't speak too fast: The impact of data bias on self-supervised speech models](https://mlsanity.com/paper/2110.07957/) (2021)
- [Emo-bias: A Large Scale Evaluation of Social Bias on Speech Emotion Recognition](https://mlsanity.com/paper/2406.05065/) (2024)
- [Adversarial Data Augmentation for Disordered Speech Recognition](https://mlsanity.com/paper/2108.00899/) (2021)
- [On the social bias of speech self-supervised models](https://mlsanity.com/paper/2406.04997/) (2024)
- [Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Models](https://mlsanity.com/paper/2408.07665/) (2024)
- [Speech Emotion Recognition using Supervised Deep Recurrent System for Mental Health Monitoring](https://mlsanity.com/paper/2208.12812/) (2022)
- [Improving fairness in speaker verification via Group-adapted Fusion Network](https://mlsanity.com/paper/2202.11323/) (2022)
- [Improving Fairness and Robustness in End-to-End Speech Recognition through unsupervised clustering](https://mlsanity.com/paper/2306.06083/) (2023)
- [Exploring data augmentation in bias mitigation against non-native-accented speech](https://mlsanity.com/paper/2312.15499/) (2023)
- [Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection](https://mlsanity.com/paper/2409.10985/) (2024)
- [Pre-trained Speech Processing Models Contain Human-Like Biases that Propagate to Speech Emotion Recognition](https://mlsanity.com/paper/2310.18877/) (2023)
- [A Contrastive Learning Approach to Mitigate Bias in Speech Models](https://mlsanity.com/paper/2406.14686/) (2024)
- [Data Augmentation via Subgroup Mixup for Improving Fairness](https://mlsanity.com/paper/2309.07110/) (2023)
- [End-to-End Continuous Speech Emotion Recognition in Real-life Customer Service Call Center Conversations](https://mlsanity.com/paper/2310.02281/) (2023)

## Built on by (most cited first, 1 of 1)

- [VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speech](https://mlsanity.com/paper/2604.17248/) (2026)

## Abstract (arXiv)

Bias in speech emotion recognition (SER) systems often stems from spurious correlations between speaker characteristics and emotional labels, leading to unfair predictions across demographic groups. Many existing debiasing methods require model-specific changes or demographic annotations, limiting their practical use. We present CO-VADA, a Confidence-Oriented Voice Augmentation Debiasing Approach that mitigates bias without modifying model architecture or relying on demographic information. CO-VADA identifies training samples that reflect bias patterns present in the training data and then applies voice conversion to alter irrelevant attributes and generate samples. These augmented samples introduce speaker variations that differ from dominant patterns in the data, guiding the model to focus more on emotion-relevant features. Our framework is compatible with various SER models and voice conversion tools, making it a scalable and practical solution for improving fairness in SER systems.