# Automatic Evaluation of Turn-taking Cues in Conversational Speech Synthesis

arXiv: https://arxiv.org/abs/2305.17971 · HTML: https://mlsanity.com/paper/2305.17971/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2023.

## Built on by (most cited first, 8 of 8)

- [Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics](https://mlsanity.com/paper/2503.01174/) (2025)
- [Applying General Turn-taking Models to Conversational Human-Robot Interaction](https://mlsanity.com/paper/2501.08946/) (2025)
- [Visual Cues Enhance Predictive Turn-Taking for Two-Party Human Interaction](https://mlsanity.com/paper/2505.21043/) (2025)
- [Target conversation extraction: Source separation using turn-taking dynamics](https://mlsanity.com/paper/2407.11277/) (2024)
- [Visual Cues Support Robust Turn-taking Prediction in Noise](https://mlsanity.com/paper/2505.22088/) (2025)
- [A Collection of Pragmatic-Similarity Judgments over Spoken Dialog Utterances](https://mlsanity.com/paper/2403.14808/) (2024)
- [Modeling Turn-Taking with Semantically Informed Gestures](https://mlsanity.com/paper/2510.19350/) (2025)
- [Audio-Visual Turn-taking Prediction in Cocktail Party Scenarios](https://mlsanity.com/paper/2609.17056/) (2026)

## Abstract (arXiv)

Turn-taking is a fundamental aspect of human communication where speakers convey their intention to either hold, or yield, their turn through prosodic cues. Using the recently proposed Voice Activity Projection model, we propose an automatic evaluation approach to measure these aspects for conversational speech synthesis. We investigate the ability of three commercial, and two open-source, Text-To-Speech (TTS) systems ability to generate turn-taking cues over simulated turns. By varying the stimuli, or controlling the prosody, we analyze the models performances. We show that while commercial TTS largely provide appropriate cues, they often produce ambiguous signals, and that further improvements are possible. TTS, trained on read or spontaneous speech, produce strong turn-hold but weak turn-yield cues. We argue that this approach, that focus on functional aspects of interaction, provides a useful addition to other important speech metrics, such as intelligibility and naturalness.