Auditing Bias and Safety in Voice AI Customer Care
Not on the map. Why?
Abstract (arXiv)
Voice AI systems increasingly mediate customer care interactions where caller presentation cues such as accent, affect, fluency, and urgency are available alongside the service request. Existing fairness and safety evaluations cover speech recognition disparities, spoken dialogue bias, and voice agent capability, but rarely treat customer care voice agents as stateful, multi turn, tool mediated systems where harm can appear as additional burden before any final denial occurs. We formalize a validation gated audit framework for such systems. The framework (i) separates native speech to speech, cascaded ASR to language model to TTS, and hybrid tool mediated architectures; (ii) uses matched service facts across controlled caller presentation conditions; (iii) validates fact invariance, presentation cues, artifacts, and acoustic measurements before inference; and (iv) records both material outcomes and path to service burden. We define the research problem, methodology, seven validation gates, a six family metric set, and claim boundaries for an active industry evaluation program. We illustrate the framework with a fully synthetic worked example of a refund dispute audit instance. Production system results are excluded from this release; public reporting is gated by the validation protocol.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 13 references that are in the map. First posted on arXiv 18 May 2026.
Builds on
7 of 13 references.
References · 7
- 2024SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Wordscited by 68
- 2026Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluencycited by 37
- 2025VoiceAgentBench: Are Voice Assistants ready for agentic tasks?cited by 19
- 2026EVA-Bench: A New End-to-end Framework for Evaluating Voice Agentscited by 18
- 2025Evaluating Bias in Spoken Dialogue LLMs for Real-World Decisions and Recommendationscited by 7
- 2026VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speechcited by 3
- 2026The Voice Behind the Words: Quantifying Intersectional Bias in SpeechLLMscited by 2
Built on by
Not cited yet.