# RetroThinker: Enabling Retrospective Thinking in Speech LLMs

arXiv: https://arxiv.org/abs/2609.11864 · HTML: https://mlsanity.com/paper/2609.11864/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning.
- Builds on 23 of its 24 references that are in the map.
- First posted on arXiv 10 Sep 2026.

## Placed in

- LLM reasoning / [Audio-language model reasoning](https://mlsanity.com/lr/audio-language-model-reasoning/)

## Builds on (most cited first, 23 of 23)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [SimPO: Simple Preference Optimization with a Reference-Free Reward](https://mlsanity.com/paper/2405.14734/) (2024)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation](https://mlsanity.com/paper/2401.08417/) (2024)
- [GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbot](https://mlsanity.com/paper/2412.02612/) (2024)
- [VoiceBench: Benchmarking LLM-Based Voice Assistants](https://mlsanity.com/paper/2410.17196/) (2024)
- [PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Models](https://mlsanity.com/paper/2602.06053/) (2026)
- [Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling](https://mlsanity.com/paper/2504.13169/) (2025)
- [STITCH: Simultaneous Thinking and Talking with Chunked Reasoning for Spoken Language Models](https://mlsanity.com/paper/2507.15375/) (2025)
- [Mini-Omni-Reasoner: Token-Level Thinking-in-Speaking in Large Speech Models](https://mlsanity.com/paper/2508.15827/) (2025)
- [SHANKS: Simultaneous Hearing and Thinking for Spoken Language Models](https://mlsanity.com/paper/2510.06917/) (2025)
- [FireRedChat: A Pluggable, Full-Duplex Voice Interaction System with Cascaded and Semi-Cascaded Implementations](https://mlsanity.com/paper/2509.06502/) (2025)
- [Can Speech LLMs Think while Listening?](https://mlsanity.com/paper/2510.07497/) (2025)
- [Understanding the Modality Gap: An Empirical Study on the Speech-Text Alignment Mechanism of Large Speech Language Models](https://mlsanity.com/paper/2510.12116/) (2025)
- [Aligning Spoken Dialogue Models from User Interactions](https://mlsanity.com/paper/2506.21463/) (2025)
- [Voice Evaluation of Reasoning Ability: Diagnosing the Modality-Induced Performance Gap](https://mlsanity.com/paper/2509.26542/) (2025)
- [ESPnet-SDS: Unified Toolkit and Demo for Spoken Dialogue Systems](https://mlsanity.com/paper/2503.08533/) (2025)
- [ChipChat: Low-Latency Cascaded Conversational Agent in MLX](https://mlsanity.com/paper/2509.00078/) (2025)

## Built on by (most cited first, 1 of 1)

- [RUBATO: Decoupled Clocks Allow Full-Duplex Models to Think as Fast as They Can](https://mlsanity.com/paper/2610.14299/) (2026)

## Abstract (arXiv)

Speech large language models (SpeechLLMs) offer reduced latency and retain paralinguistic nuances that are typically lost in cascaded automatic speech recognition (ASR) and text-based LM architectures. However, they continue to lag behind text-only LLMs on complex reasoning tasks, while real-time spoken interaction imposes strict latency constraints. Although prior works employ Chain-of-Thought (CoT) and concurrent reasoning to enhance reasoning capabilities without inducing prohibitive delays, an inherent accuracy-latency trade-off persists. In this paper, we investigate whether a streaming SpeechLLM can dynamically revise its reasoning traces on the fly. We introduce RetroThinker, a multi-stage post-training framework that equips the Moshi model to self-verify and forward-correct CoT steps during inference. RetroThinker combines supervised fine-tuning (SFT) on curated retrospective thinking data with length-based direct preference optimization (DPO) to optimize retrospective during early reasoning (i.e., reasoning concurrently while the user speaks). Evaluated on the GSM8K benchmark, RetroThinker significantly improves the accuracy-latency trade-off over non-retrospective baselines, achieving an 11% absolute accuracy gain at a comparable latency.