# Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains

arXiv: https://arxiv.org/abs/2505.16552 · HTML: https://mlsanity.com/paper/2505.16552/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 33 of its 40 references that are in the map.
- First posted on arXiv 22 May 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 30 of 33)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [PAL: Program-aided Language Models](https://mlsanity.com/paper/2211.10435/) (2022)
- [LIMO: Less is More for Reasoning](https://mlsanity.com/paper/2502.03387/) (2025)
- [Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models](https://mlsanity.com/paper/2503.16419/) (2025)
- [Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs](https://mlsanity.com/paper/2503.01307/) (2025)
- [Solving General Arithmetic Word Problems](https://mlsanity.com/paper/1608.01413/) (2016)
- [Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking](https://mlsanity.com/paper/2403.09629/) (2024)
- [Chain of Draft: Thinking Faster by Writing Less](https://mlsanity.com/paper/2502.18600/) (2025)
- [VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks](https://mlsanity.com/paper/2504.05118/) (2025)
- [Think before you speak: Training Language Models With Pause Tokens](https://mlsanity.com/paper/2310.02226/) (2023)
- [TokenSkip: Controllable Chain-of-Thought Compression in LLMs](https://mlsanity.com/paper/2502.12067/) (2025)
- [Reasoning with Latent Thoughts: On the Power of Looped Transformers](https://mlsanity.com/paper/2502.17416/) (2025)
- [From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://mlsanity.com/paper/2405.14838/) (2024)
- [CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation](https://mlsanity.com/paper/2502.21074/) (2025)
- [Dynamic Early Exit in Reasoning Models](https://mlsanity.com/paper/2504.15895/) (2025)
- [Compressed Chain of Thought: Efficient Reasoning Through Dense Representations](https://mlsanity.com/paper/2412.13171/) (2024)
- [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://mlsanity.com/paper/2311.01460/) (2023)
- [Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching](https://mlsanity.com/paper/2503.05179/) (2025)
- [SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs](https://mlsanity.com/paper/2502.12134/) (2025)
- [Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought](https://mlsanity.com/paper/2501.04682/) (2025)
- [Efficient Reasoning Models: A Survey](https://mlsanity.com/paper/2504.10903/) (2025)

## Built on by (most cited first, 30 of 85)

- [MemGen: Weaving Generative Latent Memory for Self-Evolving Agents](https://mlsanity.com/paper/2509.24704/) (2025)
- [SIM-CoT: Supervised Implicit Chain-of-Thought](https://mlsanity.com/paper/2509.20317/) (2025)
- [Implicit Reasoning in Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2509.02350/) (2025)
- [The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook](https://mlsanity.com/paper/2604.02029/) (2026)
- [LLM Latent Reasoning as Chain of Superposition](https://mlsanity.com/paper/2510.15522/) (2025)
- [Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step Entropy](https://mlsanity.com/paper/2508.03346/) (2025)
- [LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinking](https://mlsanity.com/paper/2508.03440/) (2025)
- [SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMs](https://mlsanity.com/paper/2510.05069/) (2025)
- [SofT-GRPO: Surpassing Discrete-Token LLM Reinforcement Learning via Gumbel-Reparameterized Soft-Thinking Policy Optimization](https://mlsanity.com/paper/2511.06411/) (2025)
- [Render-of-Thought: Rendering Textual Chain-of-Thought as Images for Visual Latent Reasoning](https://mlsanity.com/paper/2601.14750/) (2026)
- [Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey](https://mlsanity.com/paper/2507.09662/) (2025)
- [MEMENTO: Teaching LLMs to Manage Their Own Context](https://mlsanity.com/paper/2604.09852/) (2026)
- [Parallel Test-Time Scaling for Latent Reasoning Models](https://mlsanity.com/paper/2510.07745/) (2025)
- [MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolution](https://mlsanity.com/paper/2604.26283/) (2026)
- [Residual Context Diffusion Language Models](https://mlsanity.com/paper/2601.22954/) (2026)
- [ReGuLaR: Variational Latent Reasoning Guided by Rendered Chain-of-Thought](https://mlsanity.com/paper/2601.23184/) (2026)
- [How Do Latent Reasoning Methods Perform Under Weak and Strong Supervision?](https://mlsanity.com/paper/2602.22441/) (2026)
- [AdaPonderLM: Gated Pondering Language Models with Token-Wise Adaptive Depth](https://mlsanity.com/paper/2603.01914/) (2026)
- [Latent-GRPO: Group Relative Policy Optimization for Latent Reasoning](https://mlsanity.com/paper/2604.27998/) (2026)
- [LatentEvolve: Self-Evolving Test-Time Scaling in Latent Space](https://mlsanity.com/paper/2509.24771/) (2025)
- [Deep Thinking by Markov Chain of Continuous Thoughts](https://mlsanity.com/paper/2509.25020/) (2025)
- [ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spaces](https://mlsanity.com/paper/2602.11683/) (2026)
- [ThinkOmni: Lifting Textual Reasoning to Omni-modal Scenarios via Guidance Decoding](https://mlsanity.com/paper/2602.23306/) (2026)
- [SeLaR: Selective Latent Reasoning in Large Language Models](https://mlsanity.com/paper/2604.08299/) (2026)
- [Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization](https://mlsanity.com/paper/2601.21358/) (2026)
- [Towards Explainable Industrial Anomaly Detection via Knowledge-Guided Latent Reasoning](https://mlsanity.com/paper/2602.09850/) (2026)
- [EndoCoT: Scaling Endogenous Chain-of-Thought Reasoning in Diffusion Models](https://mlsanity.com/paper/2603.12252/) (2026)
- [Early Stopping for Large Reasoning Models via Confidence Dynamics](https://mlsanity.com/paper/2604.04930/) (2026)
- [Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs](https://mlsanity.com/paper/2605.02735/) (2026)
- [Thinking in Latents: Adaptive Anchor Refinement for Implicit Reasoning in LLMs](https://mlsanity.com/paper/2603.15051/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) achieve superior performance through Chain-of-Thought (CoT) reasoning, but these token-level reasoning chains are computationally expensive and inefficient. In this paper, we introduce Compressed Latent Reasoning (CoLaR), a novel framework that dynamically compresses reasoning processes in latent space through a two-stage training approach. First, during supervised fine-tuning, CoLaR extends beyond next-token prediction by incorporating an auxiliary next compressed embedding prediction objective. This process merges embeddings of consecutive tokens using a compression factor randomly sampled from a predefined range, and trains a specialized latent head to predict distributions of subsequent compressed embeddings. Second, we enhance CoLaR through reinforcement learning (RL) that leverages the latent head's non-deterministic nature to explore diverse reasoning paths and exploit more compact ones. This approach enables CoLaR to: i) perform reasoning at a dense latent level (i.e., silently), substantially reducing reasoning chain length, and ii) dynamically adjust reasoning speed at inference time by simply prompting the desired compression factor. Extensive experiments across four mathematical reasoning datasets demonstrate that CoLaR achieves 14.1% higher accuracy than latent-based baseline methods at comparable compression ratios, and reduces reasoning chain length by 53.3% with only 4.8% performance degradation compared to explicit CoT method. Moreover, when applied to more challenging mathematical reasoning tasks, our RL-enhanced CoLaR demonstrates performance gains of up to 5.4% while dramatically reducing latent reasoning chain length by 82.8%.