mlsanityData to 7 Oct 2026

Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluency

arXiv 2604.04847 ↗First posted 6 Apr 2026eess.AS cs.CLcited by 29

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

We introduce Full-Duplex-Bench-v3 (FDB-v3), a benchmark for evaluating spoken language models under naturalistic speech conditions and multi-step tool use. Unlike prior work, our dataset consists entirely of real human audio annotated for five disfluency categories, paired with scenarios requiring chained API calls across four task domains. We evaluate six model configurations -- GPT-Realtime, Gemini Live 2.5, Gemini Live 3.1, Grok, Ultravox v0.7, and a traditional Cascaded pipeline (WhisperrightarrowGPT-4orightarrowTTS) -- across accuracy, latency, and turn-taking dimensions. GPT-Realtime leads on Pass@1 (0.600) and interruption avoidance (13.5%); Gemini Live 3.1 achieves the fastest latency (4.25~s) but the lowest turn-take rate (78.0%); and the Cascaded baseline, despite a perfect turn-take rate, incurs the highest latency (10.12~s). Across all systems, self-correction handling and multi-step reasoning under hard scenarios remain the most consistent failure modes.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 13 of its 17 references that are in the map. First posted on arXiv 6 Apr 2026.

Builds on

13 of 17 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026τ-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domains14 Mar
  2. 2025VoiceAgentBench: Are Voice Assistants ready for agentic tasks?9 Oct

Other references · 11

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  3. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  4. 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 264
  5. 2025Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilitiescited by 94
  6. 2025Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Modelscited by 53
  7. 2025Full-Duplex-Bench-v2: A Multi-Turn Evaluation Framework for Duplex Dialogue Systems with an Automated Examinercited by 38
  8. 2024Speech-Copilot: Leveraging Large Language Models for Speech Processing via Task Decomposition, Modularization, and Program Generationcited by 27
  9. 2025SHANKS: Simultaneous Hearing and Thinking for Spoken Language Modelscited by 16
  10. 2025AgentChangeBench: A Multi-Dimensional Evaluation Framework for Goal-Shift Robustness in Conversational AIcited by 6
  11. 2025Enhancing Speech-to-Speech Dialogue Modeling with End-to-End Retrieval-Augmented Generationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 29.

29 papers.

Sort:

By area:

  1. 2026IHBench: Evaluating Post-Interruption Recovery in Voice Agents with Structured Workflowscited by 5
  2. 2026TurnBench: A Multi-Domain Benchmark for Turn-Taking Dynamics in Spoken Dialoguecited by 4
  3. 2026OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistantscited by 3
  4. 2026NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilitiescited by 3
  5. 2026MSI-Bench: Evaluating Multi-Speaker Voice Interaction for Collaborative AI Agentscited by 3
  6. 2026Overcoming State Inertia in Full-Duplex Spoken Language Models via Activation Steeringcited by 2
  7. 2026DuplexWorld: Can voice agents help you get through the day?cited by 2
  8. 2026MP-Bench: Evaluating Voice Agents as a Multiparty Conversation Participantcited by 2
  9. 2026Realtime-Venus: A full-duplex interaction system with asynchronous delegationcited by 2
  10. 2026SpeechGym: An Audio-Native Gym for Training Voice Agents via Reinforcement Learningcited by 1
  11. 2026DuplexSpeechBench-IFEval: Evaluating Implicit Instruction Following in Full-Duplex Voice Agentscited by 1
  12. 2026A frontend-backend architecture for tool calls in full-duplex speech modelscited by 1
17 more
  1. 2026Qwen-Audio-3.1-Realtime: Towards Reliable Agentic Voice Interactioncited by 1
  2. 2026CallScreenBench: Benchmarking Small Language Models as Phone Secretariescited by 0
  3. 2026Hear, Invoke, and Understand: A Skill-Calling Multimodal Agent for Large Audio Language Modelscited by 0
  4. 2026Should We Type or Talk to LLM Agents? A Comprehensive Study of Voice and Keyboard Input Perturbationscited by 0
  5. 2026PACE: A Playback-Aligned Context Engine for LLM-Based Full-Duplex Voice Dialoguecited by 0
  6. 2026Auditing Bias and Safety in Voice AI Customer Carecited by 0
  7. 2026Continue, Adapt, or Yield: In-Turn Adaptation to Overlapping Speech in Full-Duplex Agentscited by 0
  8. 2026τ-Elicitation: Benchmarking multi-turn entity extraction in voice agentscited by 0
  9. 2026Neither Silence nor Overlap Is Failure: Intent-Conditioned Evaluation of Turn-Taking in Full-Duplex Spoken Dialogue Modelscited by 0
  10. 2026Evaluating Real-Time Voice Agents: From Component Quality to Grounded Outcomescited by 0
  11. 2026Context Spanning: A Communication Framework for Full-Duplex Speech Models and External LLM Backendscited by 0
  12. 2026τ-Multilingual: Benchmarking Voice Agents Across Languagescited by 0
  13. 2026Almost Human, Except When It Matters: VoxParity and the Decisions a Voice Should Changecited by 0
  14. 2026DuplexAct-Bench: Broadening Full-Duplex Speech Evaluation toward Proactive Interaction across Diverse Behavioral Requirementscited by 0
  15. 2026DuplexSpeechBench-Document Grounding: Benchmarking Document Grounding and Hallucinations in Voice Agentscited by 0
  16. 2026Prompts versus Rules: Auditing and Controlling Speech Naturalness Behaviors in Voice User Simulatorscited by 0
  17. 2026DuplexAgent-RSI: Recursive Harness Improvement for Full-Duplex Voice Agent Collaborationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM
  2. 2026IndicFDB: Benchmarking Full-Duplex Voice Agents across Indian Languages
  3. 2026SteerDuplex: Steerable Duplex Speech Dialogue Models
  4. 2026Audio2Tool: Speak, Call, Act -- A Dataset for Benchmarking Speech Tool Use
  5. 2026Unit-Based Agent for Semi-Cascaded Full-Duplex Dialogue Systems
  6. 2025DuplexMamba: Enhancing Real-time Speech Conversations with Duplex and Streaming Capabilities
  7. 2025VoiceAssistant-Eval: Benchmarking AI Assistants across Listening, Speaking, and Viewing
  8. 2025VCB Bench: An Evaluation Benchmark for Audio-Grounded Large Language Model Conversational Agents