mlsanityData to 14 Oct 2026

The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoning

arXiv 2603.17837 ↗First posted 18 Mar 2026eess.AS cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

During conversational interactions, humans subconsciously engage in concurrent thinking while listening to a speaker. Although this internal cognitive processing may not always manifest as explicit linguistic structures, it is instrumental in formulating high-quality responses. Inspired by this cognitive phenomenon, we propose a novel Full-duplex LAtent and Internal Reasoning method named FLAIR that conducts latent thinking simultaneously with speech perception. Unlike conventional "thinking" mechanisms in NLP, which require post-hoc generation, our approach aligns seamlessly with spoken dialogue systems: during the user's speaking phase, it recursively feeds the latent embedding output from the previous step into the next step, enabling continuous reasoning that strictly adheres to causality without introducing additional latency. To enable this latent reasoning, we design an Evidence Lower Bound-based objective that supports efficient supervised finetuning via teacher forcing, circumventing the need for explicit reasoning annotations. Experiments demonstrate the effectiveness of this think-while-listening design, which achieves competitive results on a range of speech benchmarks. Furthermore, FLAIR robustly handles conversational dynamics and attains competitive performance on full-duplex interaction metrics.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 59 of its 77 references that are in the map. First posted on arXiv 18 Mar 2026.

Builds on

59 of 77 references.

References · 59

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  8. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  9. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  10. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  11. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  12. 2024Qwen2-Audio Technical Reportcited by 560
38 more
  1. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
  2. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 425
  3. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  4. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
  5. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 240
  6. 2017Advances in Variational Inferencecited by 236
  7. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 235
  8. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 234
  9. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 225
  10. 2024GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbotcited by 224
  11. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 222
  12. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 200
  13. 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 173
  14. 2025Scaling Latent Reasoning via Looped Language Modelscited by 165
  15. 2024Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Datasetcited by 160
  16. 2024Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLMcited by 141
  17. 2022Generative Spoken Dialogue Language Modelingcited by 126
  18. 2023Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognitioncited by 124
  19. 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 117
  20. 2025SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMscited by 116
  21. 2025MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmarkcited by 103
  22. 2025Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilitiescited by 100
  23. 2023Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLMcited by 94
  24. 2024Recent Advances in Speech Language Models: A Surveycited by 87
  25. 2025Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thoughtcited by 77
  26. 2020Streaming automatic speech recognition with the transformer modelcited by 73
  27. 2024Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agentscited by 72
  28. 2025Baichuan-Audio: A Unified Framework for End-to-End Speech Interactioncited by 68
  29. 2021Hi-Fi Multi-Speaker English TTS Datasetcited by 60
  30. 2024YODAS: Youtube-Oriented Dataset for Audio and Speechcited by 60
  31. 2025PonderLM: Pretraining Language Models to Ponder in Continuous Spacecited by 44
  32. 2025SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoningcited by 41
  33. 2024A Full-duplex Speech Dialogue Scheme Based On Large Language Modelscited by 40
  34. 2021SD-QA: Spoken Dialectal Question Answering for the Real Worldcited by 38
  35. 2025SALM-Duplex: Efficient and Direct Duplex Modeling for Speech-to-Speech Language Modelcited by 34
  36. 2025SALMONN-omni: A Standalone Speech LLM without Codec Injection for Full-duplex Conversationcited by 27
  37. 2025STITCH: Simultaneous Thinking and Talking with Chunked Reasoning for Spoken Language Modelscited by 26
  38. 2023Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognitioncited by 25

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.