mlsanityData to 14 Oct 2026

τ-Elicitation: Benchmarking multi-turn entity extraction in voice agents

arXiv 2609.13602 ↗First posted 11 Sep 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Voice agents often need to collect names, addresses, identifiers, dates, and times exactly, yet end-to-end benchmarks obscure where capture fails. We introduce τ-Elicitation, a 200-task voice benchmark spanning 10 entity types, controlled difficulty, caller realisms, and three environments. A matched text agent passes all tasks, but four voice configurations achieve robust exact success from 0.14 to 0.41. Agents increase verification for hard and unfamiliar entities and sometimes for incorrect captures, but not for their weakest caller voice; only 24 to 37 percent of verified errors are repaired. A scaffold that prescribes spelling, read-back, correction, and confirmation raises robust Pass3 by 14 to 31 points, at a cost of 21 to 28 seconds per call. Realisms such as spelling variations and restarts do not detectably affect exact success; mispronunciation increases repair effort. These results identify strategy selection and successful recovery as the central bottlenecks in exact spoken entity collection.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 21 references that are in the map. First posted on arXiv 11 Sep 2026.

Builds on

11 of 21 references.

References · 11

  1. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  2. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
  3. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
  4. 2021SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speechcited by 72
  5. 2023SpokenWOZ: A Large-Scale Speech-Text Benchmark for Spoken Task-Oriented Dialogue Agentscited by 71
  6. 2026Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluencycited by 37
  7. 2026τ-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domainscited by 36
  8. 2026EVA-Bench: A New End-to-end Framework for Evaluating Voice Agentscited by 18
  9. 2022A Slot Is Not Built in One Utterance: Spoken Language Dialogs with Sub-Slotscited by 13
  10. 2026DuplexWorld: Can voice agents help you get through the day?cited by 4
  11. 2024"I've Heard of You!": Generate Spoken Named Entity Recognition Data for Unseen Entitiescited by 2

Built on by

Not cited yet.