mlsanityData to 14 Oct 2026

VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speech

arXiv 2604.17248 ↗First posted 19 Apr 2026eess.AS cs.CL cs.SDcited by 3

Not on the map. Why?

Abstract (arXiv)

Large Audio-Language Models (LALMs) are increasingly integrated into daily applications, yet their generative biases remain underexplored. Existing speech fairness benchmarks rely on synthetic speech and Multiple-Choice Questions (MCQs), both offering a fragmented view of fairness. We propose VIBE, a framework that evaluates generative bias through open-ended tasks such as personalized recommendations, using human-recorded speech. Unlike MCQs, our method allows stereotypical associations to manifest organically without predefined options, making it easily extensible to new tasks. Evaluating 12 state-of-the-art LALMs reveals systematic biases in realistic scenarios. Both gender and accent cues trigger statistically significant distributional shifts, and bias magnitude is strongly task-dependent.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 35 of its 47 references that are in the map. First posted on arXiv 19 Apr 2026.

Builds on

35 of 47 references.

References · 35

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  4. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  5. 2025Gemma 3 Technical Reportcited by 1,824
  6. 2024A Survey on LLM-as-a-Judgecited by 1,423
  7. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  8. 2025Qwen2.5-Omni Technical Reportcited by 736
  9. 2024Qwen2-Audio Technical Reportcited by 560
  10. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 521
  11. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
  12. 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Modelscited by 219
23 more
  1. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 118
  2. 2025Step-Audio 2 Technical Reportcited by 105
  3. 2024Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Taskscited by 71
  4. 2024Examining Gender and Racial Bias in Large Vision-Language Models Using a Novel Dataset of Parallel Imagescited by 55
  5. 2025DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignmentcited by 50
  6. 2022Quantifying Social Biases Using Templates is Unreliablecited by 39
  7. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 39
  8. 2024textttModSCAN: Measuring Stereotypical Bias in Large Vision-Language Models from Vision and Language Modalitiescited by 22
  9. 2024Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Modelscited by 21
  10. 2024Emo-bias: A Large Scale Evaluation of Social Bias on Speech Emotion Recognitioncited by 20
  11. 2024Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Modelscited by 16
  12. 2025AudioTrust: Benchmarking the Multifaceted Trustworthiness of Audio Large Language Modelscited by 14
  13. 2025AHELM: A Holistic Evaluation of Audio-Language Modelscited by 14
  14. 2025Silenced Biases: The Dark Side LLMs Learned to Refusecited by 10
  15. 2025VoiceBBQ: Investigating Effect of Content and Acoustics in Social Bias of Spoken Language Modelcited by 6
  16. 2025Mitigating Subgroup Disparities in Multi-Label Speech Emotion Recognition: A Pseudo-Labeling and Unsupervised Learning Approachcited by 5
  17. 2025Unveiling Biases while Embracing Sustainability: Assessing the Dual Challenges of Automatic Speech Recognition Systemscited by 4
  18. 2023Debiased Automatic Speech Recognition for Dysarthric Speech via Sample Reweighting with Sample Affinity Testcited by 3
  19. 2025EMO-Debias: Benchmarking Gender Debiasing Techniques in Multi-Label Speech Emotion Recognitioncited by 3
  20. 2025How to Evaluate Automatic Speech Recognition: Comparing Different Performance and Bias Measurescited by 3
  21. 2025CantoASR: Prosody-Aware ASR-LALM Collaboration for Low-Resource Cantonesecited by 3
  22. 2024On the Role of Speech Data in Reducing Toxicity Detection Biascited by 2
  23. 2025CO-VADA: A Confidence-Oriented Voice Augmentation Debiasing Approach for Fair Speech Emotion Recognitioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Modelscited by 4
  2. 2026RedVox: Safety and Fairness Gaps in Speech Models Across Languagescited by 0
  3. 2026Auditing Bias and Safety in Voice AI Customer Carecited by 0