mlsanityData to 14 Oct 2026

Easy Turn: Integrating Acoustic and Linguistic Modalities for Robust Turn-Taking in Full-Duplex Spoken Dialogue Systems

arXiv 2509.23938 ↗First posted 28 Sep 2025cs.CL cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

Full-duplex interaction is crucial for natural human-machine communication, yet remains challenging as it requires robust turn-taking detection to decide when the system should speak, listen, or remain silent. Existing solutions either rely on dedicated turn-taking models, most of which are not open-sourced. The few available ones are limited by their large parameter size or by supporting only a single modality, such as acoustic or linguistic. Alternatively, some approaches finetune LLM backbones to enable full-duplex capability, but this requires large amounts of full-duplex data, which remain scarce in open-source form. To address these issues, we propose Easy Turn, an open-source, modular turn-taking detection model that integrates acoustic and linguistic bimodal information to predict four dialogue turn states: complete, incomplete, backchannel, and wait, accompanied by the release of Easy Turn trainset, a 1,145-hour speech dataset designed for training turn-taking detection models. Compared to existing open-source models like TEN Turn Detection and Smart Turn V2, our model achieves state-of-the-art turn-taking detection accuracy on our open-source Easy Turn testset. The data and model will be made publicly available on GitHub.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 25 of its 29 references that are in the map. First posted on arXiv 28 Sep 2025.

Builds on

25 of 29 references.

References · 25

  1. 2024Qwen2.5 Technical Reportcited by 4,942
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2025Qwen2.5-Omni Technical Reportcited by 736
  5. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  6. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  7. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 373
  8. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
  9. 2024GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbotcited by 224
  10. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
  11. 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 197
  12. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 144
13 more
  1. 2024Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLMcited by 141
  2. 2018AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scalecited by 128
  3. 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 125
  4. 2024Mini-Omni2: Towards Open-source GPT-4o with Vision, Speech and Duplex Capabilitiescited by 109
  5. 2025Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilitiescited by 100
  6. 2022Open Source MagicData-RAMC: A Rich Annotated Mandarin Conversational(RAMC) Speech Datasetcited by 40
  7. 2024SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generationcited by 33
  8. 2022Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue Systemscited by 28
  9. 2025Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generationcited by 20
  10. 2025LUCY: Linguistic Understanding and Control Yielding Early Stage of Hercited by 17
  11. 2025OSUM: Advancing Open Speech Understanding Models with Limited Resources in Academiacited by 13
  12. 2025LLM-Enhanced Dialogue Management for Full-Duplex Spoken Dialogue Systemscited by 9
  13. 2025DuplexMamba: Enhancing Real-time Speech Conversations with Duplex and Streaming Capabilitiescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

By area: