The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoning
Not on the map. Why?
Abstract (arXiv)
During conversational interactions, humans subconsciously engage in concurrent thinking while listening to a speaker. Although this internal cognitive processing may not always manifest as explicit linguistic structures, it is instrumental in formulating high-quality responses. Inspired by this cognitive phenomenon, we propose a novel Full-duplex LAtent and Internal Reasoning method named FLAIR that conducts latent thinking simultaneously with speech perception. Unlike conventional "thinking" mechanisms in NLP, which require post-hoc generation, our approach aligns seamlessly with spoken dialogue systems: during the user's speaking phase, it recursively feeds the latent embedding output from the previous step into the next step, enabling continuous reasoning that strictly adheres to causality without introducing additional latency. To enable this latent reasoning, we design an Evidence Lower Bound-based objective that supports efficient supervised finetuning via teacher forcing, circumventing the need for explicit reasoning annotations. Experiments demonstrate the effectiveness of this think-while-listening design, which achieves competitive results on a range of speech benchmarks. Furthermore, FLAIR robustly handles conversational dynamics and attains competitive performance on full-duplex interaction metrics.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 59 of its 77 references that are in the map. First posted on arXiv 18 Mar 2026.
Builds on
59 of 77 references.
References · 59
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
- 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
- 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
- 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
- 2024Qwen2-Audio Technical Reportcited by 560
38 more
- 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
- 2015MUSAN: A Music, Speech, and Noise Corpuscited by 425
- 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
- 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
- 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 240
- 2017Advances in Variational Inferencecited by 236
- 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 235
- 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 234
- 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 225
- 2024GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbotcited by 224
- 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 222
- 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 200
- 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 173
- 2025Scaling Latent Reasoning via Looped Language Modelscited by 165
- 2024Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Datasetcited by 160
- 2024Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLMcited by 141
- 2022Generative Spoken Dialogue Language Modelingcited by 126
- 2023Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognitioncited by 124
- 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 117
- 2025SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMscited by 116
- 2025MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmarkcited by 103
- 2025Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilitiescited by 100
- 2023Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLMcited by 94
- 2024Recent Advances in Speech Language Models: A Surveycited by 87
- 2025Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thoughtcited by 77
- 2020Streaming automatic speech recognition with the transformer modelcited by 73
- 2024Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agentscited by 72
- 2025Baichuan-Audio: A Unified Framework for End-to-End Speech Interactioncited by 68
- 2021Hi-Fi Multi-Speaker English TTS Datasetcited by 60
- 2024YODAS: Youtube-Oriented Dataset for Audio and Speechcited by 60
- 2025PonderLM: Pretraining Language Models to Ponder in Continuous Spacecited by 44
- 2025SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoningcited by 41
- 2024A Full-duplex Speech Dialogue Scheme Based On Large Language Modelscited by 40
- 2021SD-QA: Spoken Dialectal Question Answering for the Real Worldcited by 38
- 2025SALM-Duplex: Efficient and Direct Duplex Modeling for Speech-to-Speech Language Modelcited by 34
- 2025SALMONN-omni: A Standalone Speech LLM without Codec Injection for Full-duplex Conversationcited by 27
- 2025STITCH: Simultaneous Thinking and Talking with Chunked Reasoning for Spoken Language Modelscited by 26
- 2023Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognitioncited by 25
50 most cited of 59 All 59 (JSON)
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Audio Interaction Modelcited by 4
- 2026SALMONN-duo: Adaptive Dual-System Coordination for Full-Duplex Voice Agentscited by 1
- 2026AURAL: Adaptive Latent Reasoning with Joint Chunk for Speech Language Modelscited by 1
- 2026Acoustic-to-Text KV Compression for Full-Duplex Speech Modelscited by 0
- 2026LLMs are General Asynchronous Agentscited by 0
- 2026Full-duplex Spoken Language Modeling with Discrete Diffusioncited by 0
- 2026Full-duplex Spoken Language Modeling with Discrete Diffusion13 Oct 2026
- 2026AURAL: Adaptive Latent Reasoning with Joint Chunk for Speech Language Models1 Oct 2026
- 2026LLMs are General Asynchronous Agents28 Sep 2026
- 2026SALMONN-duo: Adaptive Dual-System Coordination for Full-Duplex Voice Agents28 Sep 2026
- 2026Acoustic-to-Text KV Compression for Full-Duplex Speech Models25 Sep 2026
- 2026Audio Interaction Model3 Jun 2026