# Easy Turn: Integrating Acoustic and Linguistic Modalities for Robust Turn-Taking in Full-Duplex Spoken Dialogue Systems

arXiv: https://arxiv.org/abs/2509.23938 · HTML: https://mlsanity.com/paper/2509.23938/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 29 references that are in the map.
- First posted on arXiv 28 Sep 2025.

## Builds on (most cited first, 25 of 25)

- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://mlsanity.com/paper/2311.07919/) (2023)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models](https://mlsanity.com/paper/2412.10117/) (2024)
- [GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbot](https://mlsanity.com/paper/2412.02612/) (2024)
- [WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognition](https://mlsanity.com/paper/2110.03370/) (2021)
- [VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction](https://mlsanity.com/paper/2501.01957/) (2025)
- [Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition](https://mlsanity.com/paper/2206.08317/) (2022)
- [Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLM](https://mlsanity.com/paper/2411.00774/) (2024)
- [AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scale](https://mlsanity.com/paper/1808.10583/) (2018)
- [WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2102.01547/) (2021)
- [Mini-Omni2: Towards Open-source GPT-4o with Vision, Speech and Duplex Capabilities](https://mlsanity.com/paper/2410.11190/) (2024)
- [Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities](https://mlsanity.com/paper/2503.04721/) (2025)
- [Open Source MagicData-RAMC: A Rich Annotated Mandarin Conversational(RAMC) Speech Dataset](https://mlsanity.com/paper/2203.16844/) (2022)
- [SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generation](https://mlsanity.com/paper/2411.18138/) (2024)
- [Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue Systems](https://mlsanity.com/paper/2205.15060/) (2022)
- [Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation](https://mlsanity.com/paper/2501.15907/) (2025)
- [LUCY: Linguistic Understanding and Control Yielding Early Stage of Her](https://mlsanity.com/paper/2501.16327/) (2025)
- [OSUM: Advancing Open Speech Understanding Models with Limited Resources in Academia](https://mlsanity.com/paper/2501.13306/) (2025)
- [LLM-Enhanced Dialogue Management for Full-Duplex Spoken Dialogue Systems](https://mlsanity.com/paper/2502.14145/) (2025)
- [DuplexMamba: Enhancing Real-time Speech Conversations with Duplex and Streaming Capabilities](https://mlsanity.com/paper/2502.11123/) (2025)

## Built on by (most cited first, 13 of 13)

- [Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMs](https://mlsanity.com/paper/2607.06540/) (2026)
- [DualTurn: Learning Turn-Taking from Dual-Channel Generative Speech Pretraining](https://mlsanity.com/paper/2603.08216/) (2026)
- [X2-Turn: Frame-Synchronous Dual-Head Modeling for Joint Streaming ASR and Turn State Prediction](https://mlsanity.com/paper/2608.10878/) (2026)
- [Same Words, Different Actions: Paired Turn-Taking Evaluation under Rewritten Dialogue Contexts](https://mlsanity.com/paper/2609.17360/) (2026)
- [A frontend-backend architecture for tool calls in full-duplex speech models](https://mlsanity.com/paper/2609.19334/) (2026)
- [Unit-Based Agent for Semi-Cascaded Full-Duplex Dialogue Systems](https://mlsanity.com/paper/2601.20230/) (2026)
- [JAL-Turn: Joint Acoustic-Linguistic Modeling for Real-Time and Robust Turn-Taking Detection in Full-Duplex Spoken Dialogue Systems](https://mlsanity.com/paper/2603.26515/) (2026)
- [UAF: A Unified Audio Front-end LLM for Full-Duplex Speech Interaction](https://mlsanity.com/paper/2604.19221/) (2026)
- [How Should LLMs Listen While Speaking? A Study of User-Stream Routing in Full-Duplex Spoken Dialogue](https://mlsanity.com/paper/2605.10199/) (2026)
- [IRAF: Interference-Resilient Adaptive Fusion for Noise-Robust End-to-End Full-Duplex Spoken Dialogue Systems](https://mlsanity.com/paper/2606.06559/) (2026)
- [Less can be More: What Aspects of Speech Drive End-of-Turn Detection](https://mlsanity.com/paper/2609.11066/) (2026)
- [FD-VAD: Semantic Endpoint Detection for Streaming Full-Duplex Speech](https://mlsanity.com/paper/2609.35791/) (2026)
- [Batched Speech Decisions Without Decoding: Single-Token Supervision Lets a Frozen LLM Hear Beyond the Transcript](https://mlsanity.com/paper/2610.02638/) (2026)

## Abstract (arXiv)

Full-duplex interaction is crucial for natural human-machine communication, yet remains challenging as it requires robust turn-taking detection to decide when the system should speak, listen, or remain silent. Existing solutions either rely on dedicated turn-taking models, most of which are not open-sourced. The few available ones are limited by their large parameter size or by supporting only a single modality, such as acoustic or linguistic. Alternatively, some approaches finetune LLM backbones to enable full-duplex capability, but this requires large amounts of full-duplex data, which remain scarce in open-source form. To address these issues, we propose Easy Turn, an open-source, modular turn-taking detection model that integrates acoustic and linguistic bimodal information to predict four dialogue turn states: complete, incomplete, backchannel, and wait, accompanied by the release of Easy Turn trainset, a 1,145-hour speech dataset designed for training turn-taking detection models. Compared to existing open-source models like TEN Turn Detection and Smart Turn V2, our model achieves state-of-the-art turn-taking detection accuracy on our open-source Easy Turn testset. The data and model will be made publicly available on GitHub.