mlsanityData to 14 Oct 2026

CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation

arXiv 2502.21074 ↗First posted 28 Feb 2025cs.CLcited by 225

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

Chain-of-Thought (CoT) reasoning enhances Large Language Models (LLMs) by encouraging step-by-step reasoning in natural language. However, leveraging a latent continuous space for reasoning may offer benefits in terms of both efficiency and robustness. Prior implicit CoT methods attempt to bypass language completely by reasoning in continuous space but have consistently underperformed compared to the standard explicit CoT approach. We introduce CODI (Continuous Chain-of-Thought via Self-Distillation), a novel training framework that effectively compresses natural language CoT into continuous space. CODI jointly trains a teacher task (Explicit CoT) and a student task (Implicit CoT), distilling the reasoning ability from language into continuous space by aligning the hidden states of a designated token. Our experiments show that CODI is the first implicit CoT approach to match the performance of explicit CoT on GSM8k at the GPT-2 scale, achieving a 3.1x compression rate and outperforming the previous state-of-the-art by 28.2% in accuracy. CODI also demonstrates robustness, generalizable to complex datasets, and interpretability. These results validate that LLMs can reason effectively not only in natural language, but also in a latent continuous space. Code is available at https://github.com/zhenyi4/codi.

Key facts. Cited by 225 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 33 of its 49 references that are in the map. First posted on arXiv 28 Feb 2025.

Builds on

33 of 49 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs17 Feb
  2. 2025Enhancing Auto-regressive Chain-of-Thought through Loop-Aligned Reasoning12 Feb
  3. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step23 May
  4. 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Models24 Apr
  5. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problems20 Feb
  6. 2022Large Language Models Are Reasoning Teachers20 Dec

Other references · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  8. 2020Knowledge Distillation: A Surveycited by 1,603
  9. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
  10. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  11. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  12. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 415
15 more
  1. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 375
  2. 2016Solving General Arithmetic Word Problemscited by 360
  3. 2023Think before you speak: Training Language Models With Pause Tokenscited by 264
  4. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 245
  5. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 244
  6. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 240
  7. 2024Compressed Chain of Thought: Efficient Reasoning Through Dense Representationscited by 201
  8. 2023What Formal Languages Can Transformers Express? A Surveycited by 124
  9. 2024Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuningcited by 96
  10. 2025Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoningcited by 91
  11. 2024Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capabilitycited by 83
  12. 2024UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Modelscited by 50
  13. 2024Deliberation in Latent Space via Differentiable Cache Augmentationcited by 26
  14. 2024In-Context Learning State Vector with Inner and Momentum Optimizationcited by 15
  15. 2021Analyzing Curriculum Learning for Sentiment Analysis along Task Difficulty, Pacing and Visualization Axescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 86; 2026: 139.

225 papers.

Sort:

By area:

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 464
  2. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
  3. 2025Dynamic Early Exit in Reasoning Modelscited by 219
  4. 2025ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learningcited by 192
  5. 2025Reasoning Models Can Be Effective Without Thinkingcited by 187
  6. 2025DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Modelscited by 158
  7. 2025Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computationcited by 133
  8. 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 109
  9. 2025Latent Visual Reasoningcited by 104
  10. 2025Efficient Reasoning Models: A Surveycited by 95
  11. 2025Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chainscited by 85
  12. 2025Monet: Reasoning in Latent Visual Space Beyond Images and Languagecited by 82
38 more
  1. 2025A Survey on Latent Reasoningcited by 81
  2. 2025SIM-CoT: Supervised Implicit Chain-of-Thoughtcited by 70
  3. 2025Efficient Inference for Large Reasoning Models: A Surveycited by 61
  4. 2025Inverse Scaling in Test-Time Computecited by 50
  5. 2025Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Spacecited by 43
  6. 2025Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lenscited by 43
  7. 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 43
  8. 2025ARM: Adaptive Reasoning Modelcited by 39
  9. 2025LLM Latent Reasoning as Chain of Superpositioncited by 39
  10. 2025LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinkingcited by 38
  11. 2025VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Modelscited by 37
  12. 2025Prolonged Reasoning Is Not All You Need: Certainty-Based Adaptive Routing for Efficient LLM/MLLM Reasoningcited by 35
  13. 2025Continuous Chain of Thought Enables Parallel Exploration and Reasoningcited by 35
  14. 2025Soft Tokens, Hard Truthscited by 35
  15. 2025Enabling Agents to Communicate Entirely in Latent Spacecited by 34
  16. 2025ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoningcited by 33
  17. 2025Hybrid Latent Reasoning via Reinforcement Learningcited by 32
  18. 2025Pruning the Unsurprising: Efficient LLM Reasoning via First-Token Surprisalcited by 32
  19. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasonercited by 32
  20. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  21. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoningcited by 29
  22. 2025FlashThink: An Early Exit Method For Efficient Reasoningcited by 28
  23. 2025System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcutscited by 27
  24. 2025Parallel Continuous Chain-of-Thought with Jacobi Iterationcited by 25
  25. 2025Sketch-in-Latents: Eliciting Unified Reasoning in MLLMscited by 25
  26. 2025Activation Steering for Chain-of-Thought Compressioncited by 24
  27. 2026Fast-ThinkAct: Efficient Vision-Language-Action Reasoning via Verbalizable Latent Planningcited by 24
  28. 2025Reasoning in the Dark: Interleaved Vision-Text Reasoning in Latent Spacecited by 23
  29. 2025SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMscited by 22
  30. 2025SofT-GRPO: Surpassing Discrete-Token LLM Reinforcement Learning via Gumbel-Reparameterized Soft-Thinking Policy Optimizationcited by 21
  31. 2026Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language Modelscited by 21
  32. 2026Render-of-Thought: Rendering Textual Chain-of-Thought as Images for Visual Latent Reasoningcited by 20
  33. 2026Generative Recursive Reasoningcited by 20
  34. 2025HAPO: Training Language Models to Reason Concisely via History-Aware Policy Optimizationcited by 19
  35. 2025Towards AI Search Paradigmcited by 19
  36. 2025Mull-Tokens: Modality-Agnostic Latent Thinkingcited by 19
  37. 2025The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysiscited by 18
  38. 2025OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking Systemcited by 18

50 most cited of 225 All 225 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
  2. 2024Expediting and Elevating Large Language Model Reasoning via Hidden Chain-of-Thought Decoding
  3. 2026D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models
  4. 2026Zipping the Thought: When and How Compressed Reasoning Data Works in LLM Post-Training
  5. 2025Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillation
  6. 2024Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
  7. 2025Eliciting Chain-of-Thought in Base LLMs via Gradient-Based Representation Optimization
  8. 2026Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression