Chain-of-Thought Reasoning in Streaming Full-Duplex End-to-End Spoken Dialogue Systems
On the map: LLM reasoning → Audio-language model reasoning
Abstract (arXiv)
Most end-to-end (E2E) spoken dialogue systems (SDS) rely on voice activity detection (VAD) for turn-taking, but VAD fails to distinguish between pauses and turn completions. Duplex SDS models address this by predicting output continuously, including silence tokens, thus removing the need for explicit VAD. However, they often have complex dual-channel architecture and lag behind cascaded models in semantic reasoning. To overcome these challenges, we propose SCoT: a Streaming Chain-of-Thought (CoT) framework for Duplex SDS, alternating between processing fixed-duration user input and generating responses in a blockwise manner. Using frame-level alignments, we create intermediate targets-aligned user transcripts and system responses for each block. Experiments show that our approach produces more coherent and interpretable responses than existing duplex methods while supporting lower-latency and overlapping interactions compared to turn-by-turn systems.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 39 of its 58 references that are in the map. First posted on arXiv 2 Oct 2025.
Builds on
39 of 58 references.
Closest ancestors · 3
Same direction, newest first
- 2025Chain-of-Thought Training for Open E2E Spoken Dialogue Systems31 May
- 2025Qwen2.5-Omni Technical Report26 Mar
- 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming29 Aug
Other references · 36
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
- 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
- 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
- 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
- 2023Simple and Controllable Music Generationcited by 480
- 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
- 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelcited by 307
- 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 264
24 more
- 2023emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representationcited by 196
- 2024Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLMcited by 135
- 2022Generative Spoken Dialogue Language Modelingcited by 125
- 2024Spirit LM: Interleaved Spoken and Written Language Modelcited by 123
- 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 116
- 2024WavChat: A Survey of Spoken Dialogue Modelscited by 88
- 2024Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agentscited by 69
- 2024Language Model Can Listen While Speakingcited by 62
- 2019ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkitcited by 60
- 2024OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversationcited by 57
- 2020TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialogcited by 55
- 2021ESPnet-SLU: Advancing Spoken Language Understanding through ESPnetcited by 53
- 2024Towards Robust Speech Representation Learning for Thousands of Languagescited by 48
- 2025Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamicscited by 45
- 2024A Full-duplex Speech Dialogue Scheme Based On Large Language Modelscited by 40
- 2024SpeechGPT-Gen: Scaling Chain-of-Information Speech Generationcited by 36
- 2024VERSA: A Versatile Evaluation Toolkit for Speech, Audio, and Musiccited by 30
- 2025OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaningcited by 27
- 2024Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Modelscited by 26
- 2024ESPnet-Codec: Comprehensive Training and Evaluation of Neural Codecs for Audio, Music, and Speechcited by 25
- 2018Investigating Speech Features for Continuous Turn-Taking Prediction Using LSTMscited by 16
- 2025ESPnet-SpeechLM: An Open Speech Language Model Toolkitcited by 9
- 2025ESPnet-SDS: Unified Toolkit and Demo for Spoken Dialogue Systemscited by 7
- 2024Enabling Real-Time Conversations with Minimal Training Costscited by 6
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoningcited by 5
- 2026Unit-Based Agent for Semi-Cascaded Full-Duplex Dialogue Systemscited by 1
- 2026TiCo: Time-Controllable Spoken Dialogue Modelcited by 1
- 2026Efficient Chain-of-Modality Reasoning via Progressive Compression for Spoken Language Modelscited by 0
- 2026Efficient Chain-of-Modality Reasoning via Progressive Compression for Spoken Language Models22 Jul 2026
- 2026TiCo: Time-Controllable Spoken Dialogue Model23 Mar 2026
- 2026The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoning18 Mar 2026
- 2026Unit-Based Agent for Semi-Cascaded Full-Duplex Dialogue Systems28 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2026S-MARC: Causal Streaming Reasoning for Full-Duplex Conversational Behavior Modeling
- 2025Enabling Conversational Behavior Reasoning Capabilities in Full-Duplex Speech
- 2025Chronological Thinking in Full-Duplex Spoken Dialogue Language Models
- 2026LTS-VoiceAgent: A Listen-Think-Speak Framework for Efficient Streaming Voice Interaction via Semantic Triggering and Incremental Reasoning
- 2026Beyond Fresh Starts: Stateful Inference for Streaming ASR in Conversational Voice Agents
- 2026BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM
- 2026Multi-Faceted Interactivity Alignment in Full-Duplex Speech Models
- 2026A frontend-backend architecture for tool calls in full-duplex speech models