# TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialog

arXiv: https://arxiv.org/abs/2010.10874 · HTML: https://mlsanity.com/paper/2010.10874/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2020.

## Built on by (most cited first, 30 of 56)

- [Generative Spoken Dialogue Language Modeling](https://mlsanity.com/paper/2203.16502/) (2022)
- [On The Landscape of Spoken Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2504.08528/) (2025)
- [WavChat: A Survey of Spoken Dialogue Models](https://mlsanity.com/paper/2411.13577/) (2024)
- [Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics](https://mlsanity.com/paper/2503.01174/) (2025)
- [Transformers in Speech Processing: A Survey](https://mlsanity.com/paper/2303.11607/) (2023)
- [Proactive Conversational Agents with Inner Thoughts](https://mlsanity.com/paper/2501.00383/) (2024)
- [Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue Systems](https://mlsanity.com/paper/2205.15060/) (2022)
- [Multilingual Turn-taking Prediction Using Voice Activity Projection](https://mlsanity.com/paper/2403.06487/) (2024)
- [Real-time and Continuous Turn-taking Prediction Using Voice Activity Projection](https://mlsanity.com/paper/2401.04868/) (2024)
- [Turn-taking and Backchannel Prediction with Acoustic and Large Language Model Fusion](https://mlsanity.com/paper/2401.14717/) (2024)
- [FlexDuo: A Pluggable System for Enabling Full-Duplex Capabilities in Speech Dialogue Systems](https://mlsanity.com/paper/2502.13472/) (2025)
- [Applying General Turn-taking Models to Conversational Human-Robot Interaction](https://mlsanity.com/paper/2501.08946/) (2025)
- [Large Language Models Know What To Say But Not When To Speak](https://mlsanity.com/paper/2410.16044/) (2024)
- [Predicting Turn-Taking and Backchannel in Human-Machine Conversations Using Linguistic, Acoustic, and Visual Signals](https://mlsanity.com/paper/2505.12654/) (2025)
- [When can I Speak? Predicting initiation points for spoken dialogue agents](https://mlsanity.com/paper/2208.03812/) (2022)
- [Gated Multimodal Fusion with Contrastive Learning for Turn-taking Prediction in Human-robot Dialogue](https://mlsanity.com/paper/2204.10172/) (2022)
- [A Noise-Robust Turn-Taking System for Real-World Dialogue Robots: A Field Experiment](https://mlsanity.com/paper/2503.06241/) (2025)
- [Speculative End-Turn Detector for Efficient Speech Chatbot Assistant](https://mlsanity.com/paper/2503.23439/) (2025)
- [Time to Talk: LLM Agents for Asynchronous Group Communication in Mafia Games](https://mlsanity.com/paper/2506.05309/) (2025)
- [Chain-of-Thought Reasoning in Streaming Full-Duplex End-to-End Spoken Dialogue Systems](https://mlsanity.com/paper/2510.02066/) (2025)
- [Optimizing Conversational Quality in Spoken Dialogue Systems with Reinforcement Learning from AI Feedback](https://mlsanity.com/paper/2601.19063/) (2026)
- [Speak or Stay Silent: Context-Aware Turn-Taking in Multi-Party Dialogue](https://mlsanity.com/paper/2603.11409/) (2026)
- [Attribution and Alignment: Effects of Local Context Repetition on Utterance Production and Comprehension in Dialogue](https://mlsanity.com/paper/2311.13061/) (2023)
- [Conversational Agents as Catalysts for Critical Thinking: Challenging Social Influence in Group Decision-making](https://mlsanity.com/paper/2503.14263/) (2025)
- [Beyond Words: Multimodal LLM Knows When to Speak](https://mlsanity.com/paper/2505.14654/) (2025)
- [DualTurn: Learning Turn-Taking from Dual-Channel Generative Speech Pretraining](https://mlsanity.com/paper/2603.08216/) (2026)
- [Response-conditioned Turn-taking Prediction](https://mlsanity.com/paper/2305.02036/) (2023)
- [Predictive Speech Recognition and End-of-Utterance Detection Towards Spoken Dialog Systems](https://mlsanity.com/paper/2409.19990/) (2024)
- [Prior Lessons of Incremental Dialogue and Robot Action Management for the Age of Language Models](https://mlsanity.com/paper/2501.00953/) (2025)
- [Thai Semantic End-of-Turn Detection for Real-Time Voice Agents](https://mlsanity.com/paper/2510.04016/) (2025)

## Abstract (arXiv)

Syntactic and pragmatic completeness is known to be important for turn-taking prediction, but so far machine learning models of turn-taking have used such linguistic information in a limited way. In this paper, we introduce TurnGPT, a transformer-based language model for predicting turn-shifts in spoken dialog. The model has been trained and evaluated on a variety of written and spoken dialog datasets. We show that the model outperforms two baselines used in prior work. We also report on an ablation study, as well as attention and gradient analyses, which show that the model is able to utilize the dialog context and pragmatic completeness for turn-taking prediction. Finally, we explore the model's potential in not only detecting, but also projecting, turn-completions.