# Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluency

arXiv: https://arxiv.org/abs/2604.04847 · HTML: https://mlsanity.com/paper/2604.04847/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 13 of its 17 references that are in the map.
- First posted on arXiv 6 Apr 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 13 of 13)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head](https://mlsanity.com/paper/2304.12995/) (2023)
- [Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities](https://mlsanity.com/paper/2503.04721/) (2025)
- [Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Models](https://mlsanity.com/paper/2507.23159/) (2025)
- [Full-Duplex-Bench-v2: A Multi-Turn Evaluation Framework for Duplex Dialogue Systems with an Automated Examiner](https://mlsanity.com/paper/2510.07838/) (2025)
- [τ-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domains](https://mlsanity.com/paper/2603.13686/) (2026)
- [Speech-Copilot: Leveraging Large Language Models for Speech Processing via Task Decomposition, Modularization, and Program Generation](https://mlsanity.com/paper/2407.09886/) (2024)
- [SHANKS: Simultaneous Hearing and Thinking for Spoken Language Models](https://mlsanity.com/paper/2510.06917/) (2025)
- [VoiceAgentBench: Are Voice Assistants ready for agentic tasks?](https://mlsanity.com/paper/2510.07978/) (2025)
- [AgentChangeBench: A Multi-Dimensional Evaluation Framework for Goal-Shift Robustness in Conversational AI](https://mlsanity.com/paper/2510.18170/) (2025)
- [Enhancing Speech-to-Speech Dialogue Modeling with End-to-End Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.00028/) (2025)

## Built on by (most cited first, 30 of 37)

- [EVA-Bench: A New End-to-end Framework for Evaluating Voice Agents](https://mlsanity.com/paper/2605.13841/) (2026)
- [TurnBench: A Multi-Domain Benchmark for Turn-Taking Dynamics in Spoken Dialogue](https://mlsanity.com/paper/2608.25218/) (2026)
- [IHBench: Evaluating Post-Interruption Recovery in Voice Agents with Structured Workflows](https://mlsanity.com/paper/2606.19595/) (2026)
- [OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistants](https://mlsanity.com/paper/2605.26485/) (2026)
- [Overcoming State Inertia in Full-Duplex Spoken Language Models via Activation Steering](https://mlsanity.com/paper/2606.11386/) (2026)
- [DuplexWorld: Can voice agents help you get through the day?](https://mlsanity.com/paper/2608.10716/) (2026)
- [Realtime-Venus: A full-duplex interaction system with asynchronous delegation](https://mlsanity.com/paper/2609.13814/) (2026)
- [NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilities](https://mlsanity.com/paper/2609.21967/) (2026)
- [MSI-Bench: Evaluating Multi-Speaker Voice Interaction for Collaborative AI Agents](https://mlsanity.com/paper/2609.24812/) (2026)
- [MP-Bench: Evaluating Voice Agents as a Multiparty Conversation Participant](https://mlsanity.com/paper/2609.13076/) (2026)
- [A frontend-backend architecture for tool calls in full-duplex speech models](https://mlsanity.com/paper/2609.19334/) (2026)
- [SpeechGym: An Audio-Native Gym for Training Voice Agents via Reinforcement Learning](https://mlsanity.com/paper/2608.26432/) (2026)
- [DuplexSpeechBench-IFEval: Evaluating Implicit Instruction Following in Full-Duplex Voice Agents](https://mlsanity.com/paper/2609.03423/) (2026)
- [Qwen-Audio-3.1-Realtime: Towards Reliable Agentic Voice Interaction](https://mlsanity.com/paper/2609.25176/) (2026)
- [CallScreenBench: Benchmarking Small Language Models as Phone Secretaries](https://mlsanity.com/paper/2608.01033/) (2026)
- [Hear, Invoke, and Understand: A Skill-Calling Multimodal Agent for Large Audio Language Models](https://mlsanity.com/paper/2608.01881/) (2026)
- [Should We Type or Talk to LLM Agents? A Comprehensive Study of Voice and Keyboard Input Perturbations](https://mlsanity.com/paper/2608.03970/) (2026)
- [PACE: A Playback-Aligned Context Engine for LLM-Based Full-Duplex Voice Dialogue](https://mlsanity.com/paper/2608.07631/) (2026)
- [Auditing Bias and Safety in Voice AI Customer Care](https://mlsanity.com/paper/2609.04206/) (2026)
- [Continue, Adapt, or Yield: In-Turn Adaptation to Overlapping Speech in Full-Duplex Agents](https://mlsanity.com/paper/2609.13117/) (2026)
- [τ-Elicitation: Benchmarking multi-turn entity extraction in voice agents](https://mlsanity.com/paper/2609.13602/) (2026)
- [Neither Silence nor Overlap Is Failure: Intent-Conditioned Evaluation of Turn-Taking in Full-Duplex Spoken Dialogue Models](https://mlsanity.com/paper/2609.27372/) (2026)
- [Evaluating Real-Time Voice Agents: From Component Quality to Grounded Outcomes](https://mlsanity.com/paper/2609.30798/) (2026)
- [Context Spanning: A Communication Framework for Full-Duplex Speech Models and External LLM Backends](https://mlsanity.com/paper/2609.33443/) (2026)
- [APEX-Voice: Can Voice Agents Complete Professional Workflows Through Full-Duplex Interaction](https://mlsanity.com/paper/2609.34973/) (2026)
- [τ-Multilingual: Benchmarking Voice Agents Across Languages](https://mlsanity.com/paper/2609.35820/) (2026)
- [Almost Human, Except When It Matters: VoxParity and the Decisions a Voice Should Change](https://mlsanity.com/paper/2609.35922/) (2026)
- [DuplexAct-Bench: Broadening Full-Duplex Speech Evaluation toward Proactive Interaction across Diverse Behavioral Requirements](https://mlsanity.com/paper/2609.39446/) (2026)
- [DuplexSpeechBench-Document Grounding: Benchmarking Document Grounding and Hallucinations in Voice Agents](https://mlsanity.com/paper/2610.00316/) (2026)
- [Conversation Is a Two-Body Problem: Dyadic Evaluation of Full-Duplex Dialogue Models](https://mlsanity.com/paper/2610.08125/) (2026)

## Abstract (arXiv)

We introduce Full-Duplex-Bench-v3 (FDB-v3), a benchmark for evaluating spoken language models under naturalistic speech conditions and multi-step tool use. Unlike prior work, our dataset consists entirely of real human audio annotated for five disfluency categories, paired with scenarios requiring chained API calls across four task domains. We evaluate six model configurations -- GPT-Realtime, Gemini Live 2.5, Gemini Live 3.1, Grok, Ultravox v0.7, and a traditional Cascaded pipeline (WhisperrightarrowGPT-4orightarrowTTS) -- across accuracy, latency, and turn-taking dimensions. GPT-Realtime leads on Pass@1 (0.600) and interruption avoidance (13.5%); Gemini Live 3.1 achieves the fastest latency (4.25~s) but the lowest turn-take rate (78.0%); and the Cascaded baseline, despite a perfect turn-take rate, incurs the highest latency (10.12~s). Across all systems, self-correction handling and multi-step reasoning under hard scenarios remain the most consistent failure modes.