Neither Silence nor Overlap Is Failure: Intent-Conditioned Evaluation of Turn-Taking in Full-Duplex Spoken Dialogue Models
Not on the map. Why?
Abstract (arXiv)
Benchmarks for full-duplex spoken dialogue models score turn-taking with binary fixed-window rules that reward immediate response or silence by completeness of the prior turn. We argue that the appropriateness of a response offset, whether delayed silence or anticipatory overlap, is conditional on the speaker's latent intent, identifiable only from that speaker's behavior. We introduce TACT, a benchmark of 9,728 episodes and 73.2 hours from five dyadic corpora; each episode carries dialogue history, a per-speaker memory profile, and an annotator-derived posterior over six intent classes. Scoring replaces binary windows with a strictly proper threshold-weighted continuous ranked probability score whose weights are intent-conditioned timing kernels fitted to human floor-transfer-offset distributions, proving boundedness, consistency, and binary reduction. Across eleven systems the best model reaches 0.47 against a human topline of 0.86, is nearly invariant to speaker profiles, and TACT agrees with human judgments at Spearman 0.81 versus 0.46 for binary metrics.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 62 references that are in the map. First posted on arXiv 23 Sep 2026.
Builds on
32 of 62 references.
References · 32
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2024GPT-4o System Cardcited by 4,467
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
- 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
- 2025Qwen2.5-Omni Technical Reportcited by 704
- 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
- 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
- 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 266
- 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
- 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 192
- 2018The Design and Implementation of XiaoIce, an Empathetic Social Chatbotcited by 172
20 more
- 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 162
- 2024Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLMcited by 135
- 2022Generative Spoken Dialogue Language Modelingcited by 125
- 2025Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilitiescited by 94
- 2024Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agentscited by 69
- 2024SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Wordscited by 67
- 2024Language Model Can Listen While Speakingcited by 62
- 2026PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Modelscited by 57
- 2020TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialogcited by 55
- 2022Voice Activity Projection: Self-supervised Learning of Turn-taking Eventscited by 54
- 2025Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Modelscited by 53
- 2025Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamicscited by 45
- 2025Full-Duplex-Bench-v2: A Multi-Turn Evaluation Framework for Duplex Dialogue Systems with an Automated Examinercited by 38
- 2025Seamless Interaction: Dyadic Audiovisual Motion Modeling and Large-Scale Datasetcited by 35
- 2026Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluencycited by 29
- 2022Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue Systemscited by 28
- 2024Multilingual Turn-taking Prediction Using Voice Activity Projectioncited by 26
- 2024Real-time and Continuous Turn-taking Prediction Using Voice Activity Projectioncited by 21
- 2025FD-Bench: A Full-Duplex Benchmarking Pipeline Designed for Full Duplex Spoken Dialogue Systemscited by 17
- 2022Turn-Taking Prediction for Natural Conversational Speechcited by 15
Built on by
Not cited yet.