# VoiceAgentEval: A Dual-Dimensional Benchmark for Expert-Level Intelligent Voice-Agent Evaluation of Xbench's Professional-Aligned Series

arXiv: https://arxiv.org/abs/2510.21244 · HTML: https://mlsanity.com/paper/2510.21244/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 24 Oct 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 17 of 17)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Kimi K2: Open Agentic Intelligence](https://mlsanity.com/paper/2507.20534/) (2025)
- [AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension](https://mlsanity.com/paper/2402.07729/) (2024)
- [VoiceBench: Benchmarking LLM-Based Voice Assistants](https://mlsanity.com/paper/2410.17196/) (2024)
- [xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluations](https://mlsanity.com/paper/2506.13651/) (2025)
- [Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.00783/) (2021)
- [URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Models](https://mlsanity.com/paper/2502.17810/) (2025)
- [Benchmarking Open-ended Audio Dialogue Understanding for Large Audio-Language Models](https://mlsanity.com/paper/2412.05167/) (2024)
- [ChatShop: Interactive Information Seeking with Language Agents](https://mlsanity.com/paper/2404.09911/) (2024)
- [Automated Survey Collection with LLM-based Conversational Agents](https://mlsanity.com/paper/2504.02891/) (2025)
- [Telephone Surveys Meet Conversational AI: Evaluating a LLM-Based Telephone Survey System at Scale](https://mlsanity.com/paper/2502.20140/) (2025)
- [Cloning a Conversational Voice AI Agent from Call,Recording Datasets for Telesales](https://mlsanity.com/paper/2509.04871/) (2025)

## Built on by (most cited first, 5 of 5)

- [UserLM-R1: Modeling Human Reasoning in User Language Models with Multi-Reward Reinforcement Learning](https://mlsanity.com/paper/2601.09215/) (2026)
- [Back to Basics: Revisiting ASR in the Age of Voice Agents](https://mlsanity.com/paper/2603.25727/) (2026)
- [CallScreenBench: Benchmarking Small Language Models as Phone Secretaries](https://mlsanity.com/paper/2608.01033/) (2026)
- [SkillEvo: Self-Renewing Evolution Gradients from Multi-Turn Interaction Feedback](https://mlsanity.com/paper/2608.13120/) (2026)
- [Inquesto Score: A reliability Protocol For Voice Agents](https://mlsanity.com/paper/2609.30514/) (2026)

## Abstract (arXiv)

We propose OutboundEval, a comprehensive benchmark for evaluating large language models (LLMs) in expert-level intelligent outbound calling scenarios. Unlike existing methods that suffer from three key limitations - insufficient dataset diversity and category coverage, unrealistic user simulation, and inaccurate evaluation metrics - OutboundEval addresses these issues through a structured framework. First, we design a benchmark spanning six major business domains and 30 representative sub-scenarios, each with scenario-specific process decomposition, weighted scoring, and domain-adaptive metrics. Second, we develop a large-model-driven User Simulator that generates diverse, persona-rich virtual users with realistic behaviors, emotional variability, and communication styles, providing a controlled yet authentic testing environment. Third, we introduce a dynamic evaluation method that adapts to task variations, integrating automated and human-in-the-loop assessment to measure task execution accuracy, professional knowledge application, adaptability, and user experience quality. Experiments on 12 state-of-the-art LLMs reveal distinct trade-offs between expert-level task completion and interaction fluency, offering practical insights for building reliable, human-like outbound AI systems. OutboundEval establishes a practical, extensible, and domain-oriented standard for benchmarking LLMs in professional applications.