mlsanityData to 14 Oct 2026

Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities

arXiv 2503.04721 ↗First posted 6 Mar 2025cs.CL eess.AScited by 100

Not on the map. Why?

Abstract (arXiv)

Spoken dialogue modeling poses challenges beyond text-based language modeling, requiring real-time interaction, turn-taking, and backchanneling. While most Spoken Dialogue Models (SDMs) operate in half-duplex mode-processing one turn at a time - emerging full-duplex SDMs can listen and speak simultaneously, enabling more natural conversations. However, current evaluations remain limited, focusing mainly on turn-based metrics or coarse corpus-level analyses. To address this, we introduce Full-Duplex-Bench, a benchmark that systematically evaluates key interactive behaviors: pause handling, backchanneling, turn-taking, and interruption management. Our framework uses automatic metrics for consistent, reproducible assessment and provides a fair, fast evaluation setup. By releasing our benchmark and code, we aim to advance spoken dialogue modeling and foster the development of more natural and engaging SDMs.

Key facts. Cited by 100 papers in the map (data through 14 Oct 2026). Builds on 38 of its 56 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

38 of 56 references.

References · 38

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  3. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  4. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  5. 2024AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehensioncited by 220
  6. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 200
  7. 2024VITA: Towards Open-Source Interactive Omni Multimodal LLMcited by 173
  8. 2024Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLMcited by 141
  9. 2022Generative Spoken Dialogue Language Modelingcited by 126
  10. 2020The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modelingcited by 101
  11. 2023Textually Pretrained Speech Language Modelscited by 101
  12. 2023Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speechcited by 101
26 more
  1. 2023Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLMcited by 94
  2. 2024WavChat: A Survey of Spoken Dialogue Modelscited by 90
  3. 2025MinMo: A Multimodal Large Language Model for Seamless Voice Interactioncited by 79
  4. 2022SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilitiescited by 77
  5. 2024Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agentscited by 72
  6. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 71
  7. 2024Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Taskscited by 71
  8. 2024SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Wordscited by 68
  9. 2024OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversationcited by 60
  10. 2021LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speechcited by 47
  11. 2025Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamicscited by 47
  12. 2024Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversationscited by 45
  13. 2022On the Utility of Self-supervised Models for Prosody-related Taskscited by 42
  14. 2022IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian languagescited by 40
  15. 2024A Full-duplex Speech Dialogue Scheme Based On Large Language Modelscited by 40
  16. 2024CrisperWhisper: Accurate Timestamps on Verbatim Speech Transcriptionscited by 34
  17. 2024SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generationcited by 33
  18. 2024Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Modelscited by 26
  19. 2023Paralinguistics-Enhanced Large Language Modeling of Spoken Dialoguecited by 25
  20. 2024Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedbackcited by 23
  21. 2022DUAL: Discrete Spoken Unit Adaptive Learning for Textless Spoken Question Answeringcited by 15
  22. 2024Large Language Models Know What To Say But Not When To Speakcited by 11
  23. 2023GSQA: An End-to-End Model for Generative Spoken Question Answeringcited by 9
  24. 2024Enabling Real-Time Conversations with Minimal Training Costscited by 6
  25. 2024Can LLMs Understand the Implication of Emphasized Sentences in Dialogue?cited by 5
  26. 2025Real-Time Textless Dialogue Generationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 86.

100 papers.

Sort:

By area:

  1. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 118
  2. 2026PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Modelscited by 63
  3. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 39
  4. 2026Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluencycited by 37
  5. 2026τ-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domainscited by 36
  6. 2025MTR-DuplexBench: Towards a Comprehensive Evaluation of Multi-Round Conversations for Full-Duplex Speech Language Modelscited by 22
  7. 2025SHANKS: Simultaneous Hearing and Thinking for Spoken Language Modelscited by 19
  8. 2025FLEXI: Benchmarking Full-duplex Human-LLM Speech Interactioncited by 15
  9. 2025Easy Turn: Integrating Acoustic and Linguistic Modalities for Robust Turn-Taking in Full-Duplex Spoken Dialogue Systemscited by 13
  10. 2025Game-Time: Evaluating Temporal Dynamics in Spoken Language Modelscited by 13
  11. 2026SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Modelscited by 13
  12. 2026MoshiRAG: Asynchronous Knowledge Retrieval for Full-Duplex Speech Language Modelscited by 12
38 more
  1. 2025FLM-Audio: Natural Monologues Improves Native Full-Duplex Chatbots via Dual Trainingcited by 11
  2. 2025From Turn-Taking to Synchronous Dialogue: A Survey of Full-Duplex Spoken Language Modelscited by 10
  3. 2026Multi-Faceted Interactivity Alignment in Full-Duplex Speech Modelscited by 10
  4. 2026DuplexCascade: Full-Duplex Speech-to-Speech Dialogue with VAD-Free Cascaded ASR-LLM-TTS Pipeline and Micro-Turn Optimizationcited by 7
  5. 2026Raon-Speech Technical Reportcited by 7
  6. 2026Toward Native Multimodal Modeling: A Roadmapcited by 7
  7. 2026VAmoS Bench: Voice Agent Simulation Benchcited by 7
  8. 2026TurnBench: A Multi-Domain Benchmark for Turn-Taking Dynamics in Spoken Dialoguecited by 7
  9. 2026The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoningcited by 6
  10. 2026Instruct-FD: Can Your Full-Duplex Speech System Follow Turn-Taking Instructions?cited by 6
  11. 2025EgoMem: Lifelong Memory Agent for Full-duplex Omnimodal Modelscited by 5
  12. 2026The ICASSP 2026 HumDial Challenge: Benchmarking Human-like Spoken Dialogue Systems in the LLM Eracited by 5
  13. 2026EchoChain: A Full-Duplex Benchmark for State-Update Reasoning Under Interruptionscited by 5
  14. 2026IHBench: Evaluating Post-Interruption Recovery in Voice Agents with Structured Workflowscited by 5
  15. 2026Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMscited by 5
  16. 2026OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistantscited by 4
  17. 2026Overcoming State Inertia in Full-Duplex Spoken Language Models via Activation Steeringcited by 4
  18. 2026DuplexWorld: Can voice agents help you get through the day?cited by 4
  19. 2026NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilitiescited by 4
  20. 2026MSI-Bench: Evaluating Multi-Speaker Voice Interaction for Collaborative AI Agentscited by 4
  21. 2026ASPIRin: Action Space Projection for Interactivity-Optimized Reinforcement Learning in Full-Duplex Speech Language Modelscited by 3
  22. 2026Omni-DuplexEval: Evaluating Real-time Duplex Omni-modal Interactioncited by 3
  23. 2026M3-DuplexBench: A Multi-Turn, Multilingual, Multidomain Benchmark for Full-Duplex Spoken Dialogue Modelscited by 3
  24. 2026X2-Turn: Frame-Synchronous Dual-Head Modeling for Joint Streaming ASR and Turn State Predictioncited by 3
  25. 2025EMO-Reasoning: Benchmarking Emotional Reasoning Capabilities in Spoken Dialogue Systemscited by 2
  26. 2025VoxRole: A Comprehensive Benchmark for Evaluating Speech-Based Role-Playing Agentscited by 2
  27. 2026Engagement Process: Rethinking the Temporal Interface of Action and Observationcited by 2
  28. 2026Synchronization and Turn-Taking in Full-Duplex Speech Dialogue Modelscited by 2
  29. 2026Cocktail-Talker: Multi-Speaker Dialog Modeling in Noisy Social Environments with Turn Action GRPOcited by 2
  30. 2026MP-Bench: Evaluating Voice Agents as a Multiparty Conversation Participantcited by 2
  31. 2026Same Words, Different Actions: Paired Turn-Taking Evaluation under Rewritten Dialogue Contextscited by 2
  32. 2026A frontend-backend architecture for tool calls in full-duplex speech modelscited by 2
  33. 2025TurnGuide: Enhancing Meaningful Full Duplex Spoken Interactions via Dynamic Turn-Level Text-Speech Interleavingcited by 1
  34. 2025Enabling Conversational Behavior Reasoning Capabilities in Full-Duplex Speechcited by 1
  35. 2026Human or Machine? A Preliminary Turing Test for Speech-to-Speech Interactioncited by 1
  36. 2026TiCo: Time-Controllable Spoken Dialogue Modelcited by 1
  37. 2026Dual-Axis Generative Reward Model Toward Semantic and Turn-taking Robustness in Interactive Spoken Dialogue Modelscited by 1
  38. 2026Aligning Backchannel and Dialogue Context Representations via Contrastive LLM Fine-Tuningcited by 1

50 most cited of 100 All 100 (JSON)