Easy Turn: Integrating Acoustic and Linguistic Modalities for Robust Turn-Taking in Full-Duplex Spoken Dialogue Systems
Not on the map. Why?
Abstract (arXiv)
Full-duplex interaction is crucial for natural human-machine communication, yet remains challenging as it requires robust turn-taking detection to decide when the system should speak, listen, or remain silent. Existing solutions either rely on dedicated turn-taking models, most of which are not open-sourced. The few available ones are limited by their large parameter size or by supporting only a single modality, such as acoustic or linguistic. Alternatively, some approaches finetune LLM backbones to enable full-duplex capability, but this requires large amounts of full-duplex data, which remain scarce in open-source form. To address these issues, we propose Easy Turn, an open-source, modular turn-taking detection model that integrates acoustic and linguistic bimodal information to predict four dialogue turn states: complete, incomplete, backchannel, and wait, accompanied by the release of Easy Turn trainset, a 1,145-hour speech dataset designed for training turn-taking detection models. Compared to existing open-source models like TEN Turn Detection and Smart Turn V2, our model achieves state-of-the-art turn-taking detection accuracy on our open-source Easy Turn testset. The data and model will be made publicly available on GitHub.
Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 25 of its 29 references that are in the map. First posted on arXiv 28 Sep 2025.
Builds on
25 of 29 references.
References · 25
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2025Qwen2.5-Omni Technical Reportcited by 736
- 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
- 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
- 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 373
- 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
- 2024GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbotcited by 224
- 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
- 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 197
- 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 144
13 more
- 2024Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLMcited by 141
- 2018AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scalecited by 128
- 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 125
- 2024Mini-Omni2: Towards Open-source GPT-4o with Vision, Speech and Duplex Capabilitiescited by 109
- 2025Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilitiescited by 100
- 2022Open Source MagicData-RAMC: A Rich Annotated Mandarin Conversational(RAMC) Speech Datasetcited by 40
- 2024SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generationcited by 33
- 2022Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue Systemscited by 28
- 2025Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generationcited by 20
- 2025LUCY: Linguistic Understanding and Control Yielding Early Stage of Hercited by 17
- 2025OSUM: Advancing Open Speech Understanding Models with Limited Resources in Academiacited by 13
- 2025LLM-Enhanced Dialogue Management for Full-Duplex Spoken Dialogue Systemscited by 9
- 2025DuplexMamba: Enhancing Real-time Speech Conversations with Duplex and Streaming Capabilitiescited by 5
Built on by
Citing papers per year: 2026: 13.
13 papers.
Sort:
By area:
- 2026Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMscited by 5
- 2026DualTurn: Learning Turn-Taking from Dual-Channel Generative Speech Pretrainingcited by 4
- 2026X2-Turn: Frame-Synchronous Dual-Head Modeling for Joint Streaming ASR and Turn State Predictioncited by 3
- 2026Same Words, Different Actions: Paired Turn-Taking Evaluation under Rewritten Dialogue Contextscited by 2
- 2026A frontend-backend architecture for tool calls in full-duplex speech modelscited by 2
- 2026Unit-Based Agent for Semi-Cascaded Full-Duplex Dialogue Systemscited by 1
- 2026JAL-Turn: Joint Acoustic-Linguistic Modeling for Real-Time and Robust Turn-Taking Detection in Full-Duplex Spoken Dialogue Systemscited by 1
- 2026UAF: A Unified Audio Front-end LLM for Full-Duplex Speech Interactioncited by 0
- 2026How Should LLMs Listen While Speaking? A Study of User-Stream Routing in Full-Duplex Spoken Dialoguecited by 0
- 2026IRAF: Interference-Resilient Adaptive Fusion for Noise-Robust End-to-End Full-Duplex Spoken Dialogue Systemscited by 0
- 2026Less can be More: What Aspects of Speech Drive End-of-Turn Detectioncited by 0
- 2026FD-VAD: Semantic Endpoint Detection for Streaming Full-Duplex Speechcited by 0
- 2026Batched Speech Decisions Without Decoding: Single-Token Supervision Lets a Frozen LLM Hear Beyond the Transcript2 Oct 2026
- 2026FD-VAD: Semantic Endpoint Detection for Streaming Full-Duplex Speech16 Sep 2026
- 2026A frontend-backend architecture for tool calls in full-duplex speech models16 Sep 2026
- 2026Same Words, Different Actions: Paired Turn-Taking Evaluation under Rewritten Dialogue Contexts15 Sep 2026
- 2026Less can be More: What Aspects of Speech Drive End-of-Turn Detection10 Sep 2026
- 2026X2-Turn: Frame-Synchronous Dual-Head Modeling for Joint Streaming ASR and Turn State Prediction11 Aug 2026
- 2026Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMs7 Jul 2026
- 2026IRAF: Interference-Resilient Adaptive Fusion for Noise-Robust End-to-End Full-Duplex Spoken Dialogue Systems4 Jun 2026
- 2026How Should LLMs Listen While Speaking? A Study of User-Stream Routing in Full-Duplex Spoken Dialogue11 May 2026
- 2026UAF: A Unified Audio Front-end LLM for Full-Duplex Speech Interaction21 Apr 2026
- 2026JAL-Turn: Joint Acoustic-Linguistic Modeling for Real-Time and Robust Turn-Taking Detection in Full-Duplex Spoken Dialogue Systems27 Mar 2026
- 2026DualTurn: Learning Turn-Taking from Dual-Channel Generative Speech Pretraining9 Mar 2026
1 more
- 2026Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMscited by 5
- 2026X2-Turn: Frame-Synchronous Dual-Head Modeling for Joint Streaming ASR and Turn State Predictioncited by 3
- 2026Same Words, Different Actions: Paired Turn-Taking Evaluation under Rewritten Dialogue Contextscited by 2
- 2026JAL-Turn: Joint Acoustic-Linguistic Modeling for Real-Time and Robust Turn-Taking Detection in Full-Duplex Spoken Dialogue Systemscited by 1
- 2026UAF: A Unified Audio Front-end LLM for Full-Duplex Speech Interactioncited by 0
- 2026How Should LLMs Listen While Speaking? A Study of User-Stream Routing in Full-Duplex Spoken Dialoguecited by 0
- 2026IRAF: Interference-Resilient Adaptive Fusion for Noise-Robust End-to-End Full-Duplex Spoken Dialogue Systemscited by 0
- 2026Less can be More: What Aspects of Speech Drive End-of-Turn Detectioncited by 0
- 2026FD-VAD: Semantic Endpoint Detection for Streaming Full-Duplex Speechcited by 0
- 2026Batched Speech Decisions Without Decoding: Single-Token Supervision Lets a Frozen LLM Hear Beyond the Transcriptcited by 0
- 2026Batched Speech Decisions Without Decoding: Single-Token Supervision Lets a Frozen LLM Hear Beyond the Transcript2 Oct 2026
- 2026FD-VAD: Semantic Endpoint Detection for Streaming Full-Duplex Speech16 Sep 2026
- 2026Same Words, Different Actions: Paired Turn-Taking Evaluation under Rewritten Dialogue Contexts15 Sep 2026
- 2026Less can be More: What Aspects of Speech Drive End-of-Turn Detection10 Sep 2026
- 2026X2-Turn: Frame-Synchronous Dual-Head Modeling for Joint Streaming ASR and Turn State Prediction11 Aug 2026
- 2026Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMs7 Jul 2026
- 2026IRAF: Interference-Resilient Adaptive Fusion for Noise-Robust End-to-End Full-Duplex Spoken Dialogue Systems4 Jun 2026
- 2026How Should LLMs Listen While Speaking? A Study of User-Stream Routing in Full-Duplex Spoken Dialogue11 May 2026
- 2026UAF: A Unified Audio Front-end LLM for Full-Duplex Speech Interaction21 Apr 2026
- 2026JAL-Turn: Joint Acoustic-Linguistic Modeling for Real-Time and Robust Turn-Taking Detection in Full-Duplex Spoken Dialogue Systems27 Mar 2026