# TurnBench: A Multi-Domain Benchmark for Turn-Taking Dynamics in Spoken Dialogue

arXiv: https://arxiv.org/abs/2608.25218 · HTML: https://mlsanity.com/paper/2608.25218/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 42 references that are in the map.
- First posted on arXiv 25 Aug 2026.

## Builds on (most cited first, 19 of 19)

- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [pyannote.audio: neural building blocks for speaker diarization](https://mlsanity.com/paper/1911.01255/) (2019)
- [Generative Spoken Dialogue Language Modeling](https://mlsanity.com/paper/2203.16502/) (2022)
- [Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities](https://mlsanity.com/paper/2503.04721/) (2025)
- [The fifth 'CHiME' Speech Separation and Recognition Challenge: Dataset, task and baselines](https://mlsanity.com/paper/1803.10609/) (2018)
- [Voice Activity Projection: Self-supervised Learning of Turn-taking Events](https://mlsanity.com/paper/2205.09812/) (2022)
- [Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics](https://mlsanity.com/paper/2503.01174/) (2025)
- [Full-Duplex-Bench-v2: A Multi-Turn Evaluation Framework for Duplex Dialogue Systems with an Automated Examiner](https://mlsanity.com/paper/2510.07838/) (2025)
- [Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluency](https://mlsanity.com/paper/2604.04847/) (2026)
- [The Edinburgh International Accents of English Corpus: Towards the Democratization of English ASR](https://mlsanity.com/paper/2303.18110/) (2023)
- [Multilingual Turn-taking Prediction Using Voice Activity Projection](https://mlsanity.com/paper/2403.06487/) (2024)
- [Real-time and Continuous Turn-taking Prediction Using Voice Activity Projection](https://mlsanity.com/paper/2401.04868/) (2024)
- [NOTSOFAR-1 Challenge: New Datasets, Baseline, and Tasks for Distant Meeting Transcription](https://mlsanity.com/paper/2401.08887/) (2024)
- [DiPCo -- Dinner Party Corpus](https://mlsanity.com/paper/1909.13447/) (2019)
- [CASPER: A Large Scale Spontaneous Speech Dataset](https://mlsanity.com/paper/2506.00267/) (2025)
- [Streaming Endpointer for Spoken Dialogue using Neural Audio Codecs and Label-Delayed Training](https://mlsanity.com/paper/2506.07081/) (2025)
- [Endpoint Anticipation for Low-Latency Spoken Dialogue](https://mlsanity.com/paper/2606.13450/) (2026)
- [ANCHOR: Autoregressive Non-intrusive Chunk-Ordered Refinement for Joint Multi-Resolution Speech Quality Modeling](https://mlsanity.com/paper/2606.10233/) (2026)

## Built on by (most cited first, 7 of 7)

- [Same Words, Different Actions: Paired Turn-Taking Evaluation under Rewritten Dialogue Contexts](https://mlsanity.com/paper/2609.17360/) (2026)
- [Continue, Adapt, or Yield: In-Turn Adaptation to Overlapping Speech in Full-Duplex Agents](https://mlsanity.com/paper/2609.13117/) (2026)
- [Controlling Backchannels in Streamable Full-duplex Models](https://mlsanity.com/paper/2609.29418/) (2026)
- [IndicFDB: Benchmarking Full-Duplex Voice Agents across Indian Languages](https://mlsanity.com/paper/2609.31967/) (2026)
- [FD-VAD: Semantic Endpoint Detection for Streaming Full-Duplex Speech](https://mlsanity.com/paper/2609.35791/) (2026)
- [People Making Conversation: Evaluating How Naturally Speech Models Converse](https://mlsanity.com/paper/2610.12802/) (2026)
- [MultiPartyTurnBench: Benchmarking Floor Control in Multi-Party Spoken Dialogue](https://mlsanity.com/paper/2610.20172/) (2026)

## Abstract (arXiv)

Speakers in natural conversation take turns speaking and listening, deciding in real time when to take, hold, or yield the floor. However, turn-taking evaluation remains limited due to the lack of a consistent, linguistically grounded evaluation protocol and hand-annotated data covering diverse conversation types. To address this, we present TurnBench, a multi-domain benchmark that pairs a 30-hour, hand-labeled corpus of dyadic human conversation with a standardized evaluation protocol for end-of-turn and interruption detection. We set conversation type as a controllable experimental variable, covering six distinct interaction styles, and triple-annotate each conversation. Benchmarking 14 heterogeneous turn-taking systems, we find end-of-turn recall stable across types, while interruption false positives are strongly type-dependent and concentrated in backchannel-dense interaction styles. Although in smooth floor transfers human listeners begin speaking a median 151 ms before the current turn ends, no current system performs equivalently without incurring excessive false positives. We release our corpus, a 104-hour training set, and a public leaderboard with an interactive dataset viewer at https://turnbench.sesame.com.