# TurnNat: Automatic Evaluation of Turn-Taking Naturalness in Dyadic Spoken Dialogue

arXiv: https://arxiv.org/abs/2607.01345 · HTML: https://mlsanity.com/paper/2607.01345/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 1 Jul 2026.

## Builds on (most cited first, 22 of 22)

- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming](https://mlsanity.com/paper/2408.16725/) (2024)
- [LLaMA-Omni: Seamless Speech Interaction with Large Language Models](https://mlsanity.com/paper/2409.06666/) (2024)
- [Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLM](https://mlsanity.com/paper/2411.00774/) (2024)
- [Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities](https://mlsanity.com/paper/2503.04721/) (2025)
- [WavChat: A Survey of Spoken Dialogue Models](https://mlsanity.com/paper/2411.13577/) (2024)
- [Voice Activity Projection: Self-supervised Learning of Turn-taking Events](https://mlsanity.com/paper/2205.09812/) (2022)
- [TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialog](https://mlsanity.com/paper/2010.10874/) (2020)
- [Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Models](https://mlsanity.com/paper/2507.23159/) (2025)
- [Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics](https://mlsanity.com/paper/2503.01174/) (2025)
- [Seamless Interaction: Dyadic Audiovisual Motion Modeling and Large-Scale Dataset](https://mlsanity.com/paper/2506.22554/) (2025)
- [Multilingual Turn-taking Prediction Using Voice Activity Projection](https://mlsanity.com/paper/2403.06487/) (2024)
- [Real-time and Continuous Turn-taking Prediction Using Voice Activity Projection](https://mlsanity.com/paper/2401.04868/) (2024)
- [Turn-taking and Backchannel Prediction with Acoustic and Large Language Model Fusion](https://mlsanity.com/paper/2401.14717/) (2024)
- [NTPP: Generative Speech Language Modeling for Dual-Channel Spoken Dialogue via Next-Token-Pair Prediction](https://mlsanity.com/paper/2506.00975/) (2025)
- [Mitigating Response Delays in Free-Form Conversations with LLM-powered Intelligent Virtual Agents](https://mlsanity.com/paper/2507.22352/) (2025)
- [A Mixed-Methods Approach to Understanding User Trust after Voice Assistant Failures](https://mlsanity.com/paper/2303.00164/) (2023)
- [Predicting Turn-Taking and Backchannel in Human-Machine Conversations Using Linguistic, Acoustic, and Visual Signals](https://mlsanity.com/paper/2505.12654/) (2025)
- [Visual Cues Enhance Predictive Turn-Taking for Two-Party Human Interaction](https://mlsanity.com/paper/2505.21043/) (2025)
- [DualTurn: Learning Turn-Taking from Dual-Channel Generative Speech Pretraining](https://mlsanity.com/paper/2603.08216/) (2026)
- [Neural Generation of Dialogue Response Timings](https://mlsanity.com/paper/2005.09128/) (2020)
- [Prompt-Guided Turn-Taking Prediction](https://mlsanity.com/paper/2506.21191/) (2025)

## Built on by (most cited first, 1 of 1)

- [GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks](https://mlsanity.com/paper/2607.05369/) (2026)

## Abstract (arXiv)

Turn-taking naturalness is central to full-duplex spoken dialogue systems, yet its automatic evaluation remains limited. Existing evaluations often rely on human judgments or behavior-specific timing metrics, making it difficult to compare heterogeneous timing failures within a unified framework. We propose TurnNat, a likelihood-based framework for automatic turn-taking naturalness evaluation in two-channel spoken dialogue. A causal turn-taking prediction model trained on natural conversations estimates future two-speaker voice-activity states, and the negative log-likelihood (NLL) of the observed future activity measures timing atypicality. TurnNat pools frame-level NLLs over turn-taking boundary units (TBUs) extracted from utterance onsets and offsets, and aggregates mean and tail TBU scores into a dialogue-level naturalness score. We further construct a controlled perturbation benchmark of paired natural and perturbed dialogue clips, validated by human naturalness judgments. Experiments on this benchmark show that TurnNat successfully identifies unnatural turn-taking perturbations across heterogeneous timing failures.