mlsanityData to 7 Oct 2026

VoiceAgentEval: A Dual-Dimensional Benchmark for Expert-Level Intelligent Voice-Agent Evaluation of Xbench's Professional-Aligned Series

arXiv 2510.21244 ↗First posted 24 Oct 2025cs.AIcited by 4

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

We propose OutboundEval, a comprehensive benchmark for evaluating large language models (LLMs) in expert-level intelligent outbound calling scenarios. Unlike existing methods that suffer from three key limitations - insufficient dataset diversity and category coverage, unrealistic user simulation, and inaccurate evaluation metrics - OutboundEval addresses these issues through a structured framework. First, we design a benchmark spanning six major business domains and 30 representative sub-scenarios, each with scenario-specific process decomposition, weighted scoring, and domain-adaptive metrics. Second, we develop a large-model-driven User Simulator that generates diverse, persona-rich virtual users with realistic behaviors, emotional variability, and communication styles, providing a controlled yet authentic testing environment. Third, we introduce a dynamic evaluation method that adapts to task variations, integrating automated and human-in-the-loop assessment to measure task execution accuracy, professional knowledge application, adaptability, and user experience quality. Experiments on 12 state-of-the-art LLMs reveal distinct trade-offs between expert-level task completion and interaction fluency, offering practical insights for building reliable, human-like outbound AI systems. OutboundEval establishes a practical, extensible, and domain-oriented standard for benchmarking LLMs in professional applications.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 17 of its 31 references that are in the map. First posted on arXiv 24 Oct 2025.

Builds on

17 of 31 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Cloning a Conversational Voice AI Agent from Call,Recording Datasets for Telesales5 Sep
  2. 2025Kimi K2: Open Agentic Intelligence28 Jul
  3. 2025Automated Survey Collection with LLM-based Conversational Agents2 Apr
  4. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun
  5. 2024ChatShop: Interactive Information Seeking with Language Agents15 Apr

Other references · 12

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  4. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  5. 2024DeepSeek-V3 Technical Reportcited by 724
  6. 2024AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehensioncited by 215
  7. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 192
  8. 2025xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluationscited by 118
  9. 2021Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systemscited by 73
  10. 2025URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Modelscited by 29
  11. 2024Benchmarking Open-ended Audio Dialogue Understanding for Large Audio-Language Modelscited by 28
  12. 2025Telephone Surveys Meet Conversational AI: Evaluating a LLM-Based Telephone Survey System at Scalecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Benchmarking LLM Judges for Voice-Agent Evaluation: Reliability, Calibration, and Human Oversight
  2. 2026Talk2Agent: Benchmarking Voice Interfaces for Text Agents
  3. 2026τ-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domains
  4. 2025VoiceAgentBench: Are Voice Assistants ready for agentic tasks?
  5. 2025VoiceAssistant-Eval: Benchmarking AI Assistants across Listening, Speaking, and Viewing
  6. 2026Evaluating Real-Time Voice Agents: From Component Quality to Grounded Outcomes
  7. 2025MEDAL: A Framework for Benchmarking LLMs as Multilingual Open-Domain Dialogue Evaluators
  8. 2025VCB Bench: An Evaluation Benchmark for Audio-Grounded Large Language Model Conversational Agents