# DualTurn: Learning Turn-Taking from Dual-Channel Generative Speech Pretraining

arXiv: https://arxiv.org/abs/2603.08216 · HTML: https://mlsanity.com/paper/2603.08216/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning.
- Builds on 14 of its 22 references that are in the map.
- First posted on arXiv 9 Mar 2026.

## Placed in

- LLM reasoning / [Audio-language model reasoning](https://mlsanity.com/lr/audio-language-model-reasoning/)
- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 14 of 14)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [Generative Spoken Dialogue Language Modeling](https://mlsanity.com/paper/2203.16502/) (2022)
- [PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Models](https://mlsanity.com/paper/2602.06053/) (2026)
- [Voice Activity Projection: Self-supervised Learning of Turn-taking Events](https://mlsanity.com/paper/2205.09812/) (2022)
- [TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialog](https://mlsanity.com/paper/2010.10874/) (2020)
- [Turn-taking and Backchannel Prediction with Acoustic and Large Language Model Fusion](https://mlsanity.com/paper/2401.14717/) (2024)
- [NTPP: Generative Speech Language Modeling for Dual-Channel Spoken Dialogue via Next-Token-Pair Prediction](https://mlsanity.com/paper/2506.00975/) (2025)
- [Investigating Speech Features for Continuous Turn-Taking Prediction Using LSTMs](https://mlsanity.com/paper/1806.11461/) (2018)
- [Easy Turn: Integrating Acoustic and Linguistic Modalities for Robust Turn-Taking in Full-Duplex Spoken Dialogue Systems](https://mlsanity.com/paper/2509.23938/) (2025)
- [Yeah, Un, Oh: Continuous and Real-time Backchannel Prediction with Fine-tuning of Voice Activity Projection](https://mlsanity.com/paper/2410.15929/) (2024)
- [Streaming Endpointer for Spoken Dialogue using Neural Audio Codecs and Label-Delayed Training](https://mlsanity.com/paper/2506.07081/) (2025)
- [Two-pass Endpoint Detection for Speech Recognition](https://mlsanity.com/paper/2401.08916/) (2024)

## Built on by (most cited first, 4 of 4)

- [TurnNat: Automatic Evaluation of Turn-Taking Naturalness in Dyadic Spoken Dialogue](https://mlsanity.com/paper/2607.01345/) (2026)
- [Controlling Backchannels in Streamable Full-duplex Models](https://mlsanity.com/paper/2609.29418/) (2026)
- [XTurnix: Large-Scale Self-Supervised Turn Control through Two-State Binary Decisions](https://mlsanity.com/paper/2610.04400/) (2026)
- [MultiPartyTurnBench: Benchmarking Floor Control in Multi-Party Spoken Dialogue](https://mlsanity.com/paper/2610.20172/) (2026)

## Abstract (arXiv)

Speech-to-speech models handle turn-taking naturally but offer limited support for tool-calling or complex reasoning, while production ASR-LLM-TTS voice pipelines offer these capabilities but rely on silence timeouts, which lead to unnatural turn-taking. We present DualTurn, which narrows this gap through generative pretraining on dual-channel conversational audio. The model generates both speakers' future audio autoregressively, implicitly learning conversational dynamics without any labels, and is then fine-tuned to predict interpretable turn-taking signals that map directly to agent actions. DualTurn monitors both channels continuously, anticipating turn boundaries and producing five agent actions. On standard benchmarks, DualTurn (0.5B) outperforms both VAP on agent action prediction (wF1 0.633 vs. 0.389) and a 3.1B audio-text model on word-level turn prediction (AUC 0.930 vs. 0.880), while anticipating turn boundaries earlier with fewer interruptions.