mlsanityData to 7 Oct 2026

A Survey on Latent Reasoning

arXiv 2507.06203 ↗First posted 8 Jul 2025cs.CLcited by 71

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, especially when guided by explicit chain-of-thought (CoT) reasoning that verbalizes intermediate steps. While CoT improves both interpretability and accuracy, its dependence on natural language reasoning limits the model's expressive bandwidth. Latent reasoning tackles this bottleneck by performing multi-step inference entirely in the model's continuous hidden state, eliminating token-level supervision. To advance latent reasoning research, this survey provides a comprehensive overview of the emerging field of latent reasoning. We begin by examining the foundational role of neural network layers as the computational substrate for reasoning, highlighting how hierarchical representations support complex transformations. Next, we explore diverse latent reasoning methodologies, including activation-based recurrence, hidden state propagation, and fine-tuning strategies that compress or internalize explicit reasoning traces. Finally, we discuss advanced paradigms such as infinite-depth latent reasoning via masked diffusion models, which enable globally consistent and reversible reasoning processes. By unifying these perspectives, we aim to clarify the conceptual landscape of latent reasoning and chart future directions for research at the frontier of LLM cognition. An associated GitHub repository collecting the latest papers and repos is available at: https://github.com/multimodal-art-projection/LatentCoT-Horizon/.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 104 of its 140 references that are in the map. First posted on arXiv 8 Jul 2025.

Builds on

104 of 140 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration30 May
  2. 2025Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space19 May
  3. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformers24 Feb
  4. 2025LightThinker: Thinking Step-by-Step Compression21 Feb
  5. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach7 Feb
  6. 2025Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning5 Feb
  7. 2024Titans: Learning to Memorize at Test Time31 Dec
  8. 2024Compressed Chain of Thought: Efficient Reasoning Through Dense Representations17 Dec
  9. 2024The Unreasonable Ineffectiveness of the Deeper Layers26 Mar

Other references · 95

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  4. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  5. 2024OpenAI o1 System Cardcited by 1,220
  6. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  7. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  8. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  9. 2018Universal Transformerscited by 613
  10. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 594
  11. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 490
  12. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 483
38 more
  1. 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 373
  2. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  3. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  4. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 329
  5. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 293
  6. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  7. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  8. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 256
  9. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  10. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Stepcited by 225
  11. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 216
  12. 2022Continuous diffusion for categorical datacited by 202
  13. 2024Scaling up Masked Diffusion Models on Textcited by 195
  14. 2024Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attentioncited by 194
  15. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 178
  16. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  17. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 165
  18. 2024Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrencecited by 155
  19. 2023Likelihood-Based Diffusion Language Modelscited by 147
  20. 2025dKV-Cache: The Cache for Diffusion Language Modelscited by 140
  21. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  22. 2019On the Turing Completeness of Modern Neural Network Architecturescited by 129
  23. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 127
  24. 2024HGRN2: Gated Linear RNNs with State Expansioncited by 122
  25. 2025Test-Time Training Done Rightcited by 121
  26. 2025A Comprehensive Survey on Long Context Language Modelingcited by 113
  27. 2023A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysiscited by 88
  28. 2024Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalizationcited by 78
  29. 2024Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queriescited by 78
  30. 2024Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRAcited by 72
  31. 2024MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Seriescited by 67
  32. 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusioncited by 63
  33. 2023Bayesian Flow Networkscited by 62
  34. 2024Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning?cited by 60
  35. 2025It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimizationcited by 60
  36. 2025The Curse of Depth in Large Language Modelscited by 58
  37. 2025ATLAS: Learning to Optimally Memorize the Context at Test Timecited by 57
  38. 2024Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluationcited by 56

50 most cited of 95 All 95 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 21; 2026: 50.

71 papers.

Sort:

By area:

  1. 2025Scaling Latent Reasoning via Looped Language Modelscited by 145
  2. 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 99
  3. 2025Latent Collaboration in Multi-Agent Systemscited by 67
  4. 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 42
  5. 2025Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMscited by 41
  6. 2025Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrencecited by 41
  7. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 40
  8. 2025LLM Latent Reasoning as Chain of Superpositioncited by 37
  9. 2025Enabling Agents to Communicate Entirely in Latent Spacecited by 31
  10. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasonercited by 28
  11. 2025SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMscited by 20
  12. 2025OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking Systemcited by 18
38 more
  1. 2025SofT-GRPO: Surpassing Discrete-Token LLM Reinforcement Learning via Gumbel-Reparameterized Soft-Thinking Policy Optimizationcited by 18
  2. 2026Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language Modelscited by 14
  3. 2025Parallel Test-Time Scaling for Latent Reasoning Modelscited by 12
  4. 2026Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformerscited by 11
  5. 2026Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Taskscited by 10
  6. 2026Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal Structurecited by 9
  7. 2026SpiralFormer: Looped Transformers Can Learn Hierarchical Dependencies via Multi-Resolution Recursioncited by 9
  8. 2025LatentEvolve: Self-Evolving Test-Time Scaling in Latent Spacecited by 8
  9. 2025MeSH: Memory-as-State-Highways for Recursive Transformerscited by 8
  10. 2026ReGuLaR: Variational Latent Reasoning Guided by Rendered Chain-of-Thoughtcited by 8
  11. 2026Show, Don't Tell: Morphing Latent Reasoning into Image Generationcited by 7
  12. 2025KaVa: Latent Reasoning via Compressed KV-Cache Distillationcited by 6
  13. 2026ATLAS: Verifier-Guided Adaptive Latent Activation Steering for Efficient LLM Reasoningcited by 6
  14. 2026Likelihood-Based Reward Designs for General LLM Reasoningcited by 6
  15. 2026Prioritize the Process, Not Just the Outcome: Rewarding Latent Thought Trajectories Improves Reasoning in Looped Language Modelscited by 6
  16. 2026ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spacescited by 6
  17. 2026Are Latent Reasoning Models Easily Interpretable?cited by 6
  18. 2026Chain Of Thought Compression: A Theoretical Analysiscited by 5
  19. 2026Step-resolved data attribution for looped transformerscited by 5
  20. 2025Learning to Ponder: Adaptive Reasoning in Latent Spacecited by 4
  21. 2025Exploring System 1 and 2 communication for latent reasoning in LLMscited by 4
  22. 2026Reasoning models do not yet follow their reasoning in autonomous driving: The KITScenes LongTail Datasetcited by 4
  23. 2026Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMscited by 4
  24. 2026Beyond Visual Memory: Mechanistic Diagnostics of Latent Visual Reasoningcited by 4
  25. 2026Internalizing LLM Reasoning via Discovery and Replay of Latent Actionscited by 3
  26. 2026Latent Thoughts Tuning: Bridging Context and Reasoning with Fused Information in Latent Tokenscited by 3
  27. 2025Towards Inference-time Scaling for Continuous Space Reasoningcited by 2
  28. 2025Refine Thought: A Test-Time Inference Method for Embedding Model Reasoningcited by 2
  29. 2026Compression as Adaptation: Implicit Visual Representation with Diffusion Foundation Modelscited by 2
  30. 2026CorrectionPlanner: Self-Correction Planner with Reinforcement Learning in Autonomous Drivingcited by 2
  31. 2026Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Interventioncited by 2
  32. 2026How Transparent is DiffusionGemma?cited by 2
  33. 2025Improving Latent Reasoning in LLMs via Soft Concept Mixingcited by 1
  34. 2026A State-Transition Framework for Efficient LLM Reasoningcited by 1
  35. 2026Beyond Dense States: Sparse Transcoders as Causally Testable Operators for LLM Latent Reasoningcited by 1
  36. 2026Multimodal Latent Reasoning via Hierarchical Visual Cues Injectioncited by 1
  37. 2026Emergent Search and Backtracking in Latent Reasoning Modelscited by 1
  38. 2026Latent-DARM: Bridging Discrete Diffusion And Autoregressive Models For Reasoningcited by 1

50 most cited of 71 All 71 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
  2. 2026Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners
  3. 2026Latent Thought Flow: Efficient Latent Reasoning in Large Language Models
  4. 2024Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding
  5. 2026What Matters for Latent Reasoning with Flow Matching
  6. 2026Thinking in Latents: Adaptive Anchor Refinement for Implicit Reasoning in LLMs
  7. 2026Why Struggle with Continuous Latents? Interpretable Discrete Latent Reasoning via Rendered Compression
  8. 2026How Do Latent Reasoning Methods Perform Under Weak and Strong Supervision?