When Entanglement Lower-Bounds Disparity: Auditing and Repairing Demographic Fairness in Audio Understanding Models
Not on the map. Why?
Abstract (arXiv)
Speech technology penalizes some voices: recognition errs nearly twice as often for Black speakers, and accuracy declines for second-language accents and older speakers. We introduce TRIAD, an audit grid crossing 120 texts, 24 rendered demographic voice profiles (gender, age band, accent), and ten expressive styles via controllable text-to-speech, isolating perceived demographic attributes from content and affect. For ten open-weights encoders we define axis-fidelity functionals, principal-angle leakage between axis subspaces, and group-conditional gaps; a proposition proves that average probe disparity grows with the same aggregate voice-semantic leakage Lambda we measure, and a corollary shows that peak leakage forces worst-case disparity inside an active region. The measured mean-square probe disparity tracks Lambda (Pearson r = 0.93), and a black-box protocol exposes the same signature in two closed-source models. ORCA, an adapter combining axis-specific contrastive heads, an orthogonality penalty, and group-balanced sampling, cuts leakage 72% and roughly halves the gaps.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 68 references that are in the map. First posted on arXiv 23 Sep 2026.
Builds on
40 of 68 references.
References · 40
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2020Supervised Contrastive Learningcited by 2,803
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2016Equality of Opportunity in Supervised Learningcited by 2,000
- 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
- 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
- 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
28 more
- 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 671
- 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
- 2016Inherent Trade-Offs in the Fair Determination of Risk Scorescited by 582
- 2024Qwen2-Audio Technical Reportcited by 543
- 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
- 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 458
- 2018Learning Adversarially Fair and Transferable Representationscited by 353
- 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 293
- 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
- 2023emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representationcited by 196
- 2022Dawn of the transformer era in speech emotion recognition: closing the valence gapcited by 155
- 2022Comparative layer-wise analysis of self-supervised speech modelscited by 155
- 2019Inherent Tradeoffs in Learning Fair Representationscited by 144
- 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 124
- 2023EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesiscited by 93
- 2023Efficient Sequence Transduction by Jointly Predicting Tokens and Durationscited by 56
- 2022On the Utility of Self-supervised Models for Prosody-related Taskscited by 42
- 2025Voxtralcited by 41
- 2023The Edinburgh International Accents of English Corpus: Towards the Democratization of English ASRcited by 36
- 2025Seamless Interaction: Dyadic Audiovisual Motion Modeling and Large-Scale Datasetcited by 35
- 2021Towards Measuring Fairness in Speech Recognition: Casual Conversations Dataset Transcriptionscited by 26
- 2024Towards measuring fairness in speech recognition: Fair-Speech datasetcited by 26
- 2018Speaker-Invariant Training via Adversarial Learningcited by 23
- 2022Bias in Automated Speaker Recognitioncited by 20
- 2024Emo-bias: A Large Scale Evaluation of Social Bias on Speech Emotion Recognitioncited by 19
- 2024Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Modelscited by 15
- 2023Improving Fairness and Robustness in End-to-End Speech Recognition through unsupervised clusteringcited by 11
- 2023The Casual Conversations v2 Datasetcited by 4
Built on by
Not cited yet.