mlsanityData to 14 Oct 2026

τ-Multilingual: Benchmarking Voice Agents Across Languages

arXiv 2609.35820 ↗First posted 21 Sep 2026cs.CL cs.AI cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

English-only benchmarks expose only a narrow slice of voice-agent behavior. We introduce τ-Multilingual, extending τ-Voice to Spanish, Brazilian Portuguese, Hindi, Korean, and Mandarin with native-speaker review and evaluation of generated language and spoken output. Across 4,500 full-duplex calls and five voice configurations, Spanish, Portuguese, and Hindi remain within 3.2 task-completion points of English, but Korean and Mandarin fall by 14.7 and 8.4 points. The failure modes also vary: Korean systems miss more responses, Mandarin systems interrupt more often, and both struggle with tools and entities. Grok leads task completion but scores lowest on generation quality, motivating separate task, interaction, and generation reporting. We release language packs, validated judges, and tools for community-built multilingual voice-agent evaluation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 18 references that are in the map. First posted on arXiv 21 Sep 2026.

Builds on

12 of 18 references.

References · 12

  1. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  2. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
  3. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 338
  4. 2026Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluencycited by 37
  5. 2026τ-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domainscited by 36
  6. 2021GlobalWoZ: Globalizing MultiWoZ to Develop Multilingual Task-Oriented Dialogue Systemscited by 31
  7. 2024Speech-MASSIVE: A Multilingual Speech Dataset for SLU and Beyondcited by 20
  8. 2025VoiceAgentBench: Are Voice Assistants ready for agentic tasks?cited by 19
  9. 2026EVA-Bench: A New End-to-end Framework for Evaluating Voice Agentscited by 18
  10. 2023X-RiSAWOZ: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agentscited by 13
  11. 2025MENLO: From Preferences to Proficiency -- Evaluating and Modeling Native-like Quality Across 47 Languagescited by 3
  12. 2026M3-DuplexBench: A Multi-Turn, Multilingual, Multidomain Benchmark for Full-Duplex Spoken Dialogue Modelscited by 3

Built on by

Not cited yet.