# Chain-of-Thought Reasoning in Streaming Full-Duplex End-to-End Spoken Dialogue Systems

arXiv: https://arxiv.org/abs/2510.02066 · HTML: https://mlsanity.com/paper/2510.02066/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning.
- Builds on 39 of its 58 references that are in the map.
- First posted on arXiv 2 Oct 2025.

## Placed in

- LLM reasoning / [Audio-language model reasoning](https://mlsanity.com/lr/audio-language-model-reasoning/)

## Builds on (most cited first, 30 of 39)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [Automatic Chain of Thought Prompting in Large Language Models](https://mlsanity.com/paper/2210.03493/) (2022)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities](https://mlsanity.com/paper/2305.11000/) (2023)
- [Simple and Controllable Music Generation](https://mlsanity.com/paper/2306.05284/) (2023)
- [UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022](https://mlsanity.com/paper/2204.02152/) (2022)
- [SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model](https://mlsanity.com/paper/2502.02737/) (2025)
- [AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head](https://mlsanity.com/paper/2304.12995/) (2023)
- [Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming](https://mlsanity.com/paper/2408.16725/) (2024)
- [emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation](https://mlsanity.com/paper/2312.15185/) (2023)
- [Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLM](https://mlsanity.com/paper/2411.00774/) (2024)
- [Generative Spoken Dialogue Language Modeling](https://mlsanity.com/paper/2203.16502/) (2022)
- [Spirit LM: Interleaved Spoken and Written Language Model](https://mlsanity.com/paper/2402.05755/) (2024)
- [On The Landscape of Spoken Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2504.08528/) (2025)
- [WavChat: A Survey of Spoken Dialogue Models](https://mlsanity.com/paper/2411.13577/) (2024)
- [Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agents](https://mlsanity.com/paper/2409.15594/) (2024)
- [Language Model Can Listen While Speaking](https://mlsanity.com/paper/2408.02622/) (2024)
- [ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit](https://mlsanity.com/paper/1910.10909/) (2019)
- [OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation](https://mlsanity.com/paper/2410.17799/) (2024)
- [TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialog](https://mlsanity.com/paper/2010.10874/) (2020)
- [ESPnet-SLU: Advancing Spoken Language Understanding through ESPnet](https://mlsanity.com/paper/2111.14706/) (2021)
- [Towards Robust Speech Representation Learning for Thousands of Languages](https://mlsanity.com/paper/2407.00837/) (2024)
- [Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics](https://mlsanity.com/paper/2503.01174/) (2025)
- [A Full-duplex Speech Dialogue Scheme Based On Large Language Models](https://mlsanity.com/paper/2405.19487/) (2024)
- [SpeechGPT-Gen: Scaling Chain-of-Information Speech Generation](https://mlsanity.com/paper/2401.13527/) (2024)

## Built on by (most cited first, 5 of 5)

- [The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoning](https://mlsanity.com/paper/2603.17837/) (2026)
- [Unit-Based Agent for Semi-Cascaded Full-Duplex Dialogue Systems](https://mlsanity.com/paper/2601.20230/) (2026)
- [TiCo: Time-Controllable Spoken Dialogue Model](https://mlsanity.com/paper/2603.22267/) (2026)
- [Efficient Chain-of-Modality Reasoning via Progressive Compression for Spoken Language Models](https://mlsanity.com/paper/2607.19932/) (2026)
- [RUBATO: Decoupled Clocks Allow Full-Duplex Models to Think as Fast as They Can](https://mlsanity.com/paper/2610.14299/) (2026)

## Abstract (arXiv)

Most end-to-end (E2E) spoken dialogue systems (SDS) rely on voice activity detection (VAD) for turn-taking, but VAD fails to distinguish between pauses and turn completions. Duplex SDS models address this by predicting output continuously, including silence tokens, thus removing the need for explicit VAD. However, they often have complex dual-channel architecture and lag behind cascaded models in semantic reasoning. To overcome these challenges, we propose SCoT: a Streaming Chain-of-Thought (CoT) framework for Duplex SDS, alternating between processing fixed-duration user input and generating responses in a blockwise manner. Using frame-level alignments, we create intermediate targets-aligned user transcripts and system responses for each block. Experiments show that our approach produces more coherent and interpretable responses than existing duplex methods while supporting lower-latency and overlapping interactions compared to turn-by-turn systems.