mlsanityData to 7 Oct 2026

Chain-of-Thought Reasoning in Streaming Full-Duplex End-to-End Spoken Dialogue Systems

arXiv 2510.02066 ↗First posted 2 Oct 2025cs.CL cs.SD eess.AScited by 4

On the map: LLM reasoning → Audio-language model reasoning

Abstract (arXiv)

Most end-to-end (E2E) spoken dialogue systems (SDS) rely on voice activity detection (VAD) for turn-taking, but VAD fails to distinguish between pauses and turn completions. Duplex SDS models address this by predicting output continuously, including silence tokens, thus removing the need for explicit VAD. However, they often have complex dual-channel architecture and lag behind cascaded models in semantic reasoning. To overcome these challenges, we propose SCoT: a Streaming Chain-of-Thought (CoT) framework for Duplex SDS, alternating between processing fixed-duration user input and generating responses in a blockwise manner. Using frame-level alignments, we create intermediate targets-aligned user transcripts and system responses for each block. Experiments show that our approach produces more coherent and interpretable responses than existing duplex methods while supporting lower-latency and overlapping interactions compared to turn-by-turn systems.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 39 of its 58 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

39 of 58 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Chain-of-Thought Training for Open E2E Spoken Dialogue Systems31 May
  2. 2025Qwen2.5-Omni Technical Report26 Mar
  3. 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming29 Aug

Other references · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  5. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  6. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  7. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  8. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
  9. 2023Simple and Controllable Music Generationcited by 480
  10. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
  11. 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelcited by 307
  12. 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 264
24 more
  1. 2023emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representationcited by 196
  2. 2024Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLMcited by 135
  3. 2022Generative Spoken Dialogue Language Modelingcited by 125
  4. 2024Spirit LM: Interleaved Spoken and Written Language Modelcited by 123
  5. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 116
  6. 2024WavChat: A Survey of Spoken Dialogue Modelscited by 88
  7. 2024Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agentscited by 69
  8. 2024Language Model Can Listen While Speakingcited by 62
  9. 2019ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkitcited by 60
  10. 2024OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversationcited by 57
  11. 2020TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialogcited by 55
  12. 2021ESPnet-SLU: Advancing Spoken Language Understanding through ESPnetcited by 53
  13. 2024Towards Robust Speech Representation Learning for Thousands of Languagescited by 48
  14. 2025Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamicscited by 45
  15. 2024A Full-duplex Speech Dialogue Scheme Based On Large Language Modelscited by 40
  16. 2024SpeechGPT-Gen: Scaling Chain-of-Information Speech Generationcited by 36
  17. 2024VERSA: A Versatile Evaluation Toolkit for Speech, Audio, and Musiccited by 30
  18. 2025OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaningcited by 27
  19. 2024Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Modelscited by 26
  20. 2024ESPnet-Codec: Comprehensive Training and Evaluation of Neural Codecs for Audio, Music, and Speechcited by 25
  21. 2018Investigating Speech Features for Continuous Turn-Taking Prediction Using LSTMscited by 16
  22. 2025ESPnet-SpeechLM: An Open Speech Language Model Toolkitcited by 9
  23. 2025ESPnet-SDS: Unified Toolkit and Demo for Spoken Dialogue Systemscited by 7
  24. 2024Enabling Real-Time Conversations with Minimal Training Costscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026S-MARC: Causal Streaming Reasoning for Full-Duplex Conversational Behavior Modeling
  2. 2025Enabling Conversational Behavior Reasoning Capabilities in Full-Duplex Speech
  3. 2025Chronological Thinking in Full-Duplex Spoken Dialogue Language Models
  4. 2026LTS-VoiceAgent: A Listen-Think-Speak Framework for Efficient Streaming Voice Interaction via Semantic Triggering and Incremental Reasoning
  5. 2026Beyond Fresh Starts: Stateful Inference for Streaming ASR in Conversational Voice Agents
  6. 2026BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM
  7. 2026Multi-Faceted Interactivity Alignment in Full-Duplex Speech Models
  8. 2026A frontend-backend architecture for tool calls in full-duplex speech models