# Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Tasks

arXiv: https://arxiv.org/abs/2602.00449 · HTML: https://mlsanity.com/paper/2602.00449/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 32 of its 47 references that are in the map.
- First posted on arXiv 31 Jan 2026.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)
- Mechanistic interpretability / [Steering and interpreting LLM reasoning](https://mlsanity.com/mechinterp/steering-and-interpreting-llm-reasoning/)

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning](https://mlsanity.com/paper/2409.12183/) (2024)
- [Reasoning with Latent Thoughts: On the Power of Looped Transformers](https://mlsanity.com/paper/2502.17416/) (2025)
- [From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://mlsanity.com/paper/2405.14838/) (2024)
- [CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation](https://mlsanity.com/paper/2502.21074/) (2025)
- [A Survey on Latent Reasoning](https://mlsanity.com/paper/2507.06203/) (2025)
- [Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization](https://mlsanity.com/paper/2405.15071/) (2024)
- [Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries](https://mlsanity.com/paper/2406.12775/) (2024)
- [Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought](https://mlsanity.com/paper/2505.12514/) (2025)
- [Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation](https://mlsanity.com/paper/2410.13640/) (2024)
- [How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning](https://mlsanity.com/paper/2402.18312/) (2024)
- [The Expressive Power of Transformers with Chain of Thought](https://mlsanity.com/paper/2310.07923/) (2023)
- [Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models](https://mlsanity.com/paper/2310.14491/) (2023)
- [A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Task](https://mlsanity.com/paper/2402.11917/) (2024)
- [Understanding and Patching Compositional Reasoning in LLMs](https://mlsanity.com/paper/2402.14328/) (2024)
- [Continuous Chain of Thought Enables Parallel Exploration and Reasoning](https://mlsanity.com/paper/2505.23648/) (2025)
- [Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer](https://mlsanity.com/paper/2507.02199/) (2025)
- [Back Attention: Understanding and Enhancing Multi-Hop Reasoning in Large Language Models](https://mlsanity.com/paper/2502.10835/) (2025)
- [Do Latent Tokens Think? A Causal and Adversarial Analysis of Chain-of-Continuous-Thought](https://mlsanity.com/paper/2512.21711/) (2025)
- [An Investigation of Neuron Activation as a Unified Lens to Explain Chain-of-Thought Eliciting Arithmetic Reasoning of LLMs](https://mlsanity.com/paper/2406.12288/) (2024)
- [Complexity Control Facilitates Reasoning-Based Compositional Generalization in Transformers](https://mlsanity.com/paper/2501.08537/) (2025)
- [How Do LLMs Perform Two-Hop Reasoning in Context?](https://mlsanity.com/paper/2502.13913/) (2025)
- [Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking](https://mlsanity.com/paper/2502.20129/) (2025)
- [Do LLMs Really Think Step-by-step In Implicit Reasoning?](https://mlsanity.com/paper/2411.15862/) (2024)
- [A Formal Comparison Between Chain of Thought and Latent Thought](https://mlsanity.com/paper/2509.25239/) (2025)
- [Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Models](https://mlsanity.com/paper/2402.11900/) (2024)

## Built on by (most cited first, 12 of 12)

- [LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulation](https://mlsanity.com/paper/2602.11451/) (2026)
- [The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook](https://mlsanity.com/paper/2604.02029/) (2026)
- [Training-Free Looped Transformers](https://mlsanity.com/paper/2605.23872/) (2026)
- [Are Latent Reasoning Models Easily Interpretable?](https://mlsanity.com/paper/2604.04902/) (2026)
- [Observable Patterns Are Not Explanations: A Causal-Geometric Analysis of Latent Reasoning Models](https://mlsanity.com/paper/2606.12689/) (2026)
- [How Transparent is DiffusionGemma?](https://mlsanity.com/paper/2606.20560/) (2026)
- [MUX: Continuous Reasoning via Multiplexed Tokens](https://mlsanity.com/paper/2607.18264/) (2026)
- [Final Checkpoints Are Not Enough: Analyzing Latent Reasoning Faithfulness Along Training Trajectories](https://mlsanity.com/paper/2607.06648/) (2026)
- [Interpreting Latent CoT Reasoning as Dynamical Systems](https://mlsanity.com/paper/2607.09698/) (2026)
- [ChronoVision: Temporal Reasoning via Latent State Reconstruction](https://mlsanity.com/paper/2608.05631/) (2026)
- [Beyond the Training Horizon: Mechanisms and Limits of Length Generalization in Looped Transformers](https://mlsanity.com/paper/2609.33144/) (2026)
- [Dynamic K-Head: Difficulty-Supervised Budget Selection for Continuous Latent Reasoning](https://mlsanity.com/paper/2610.17679/) (2026)

## Abstract (arXiv)

Latent Chain-of-Thought aims to enable step-by-step computation without emitting long rationales, yet its internal mechanisms remain unclear. We study CODI, a continuous-thought teacher-student distillation model, on strictly sequential polynomial-iteration tasks with known intermediate states. Using logit-lens decoding, linear probes, attention analysis, and activation patching, we localize intermediate-state representations and trace how they are routed to the final readout. In short-horizon, low-hop tasks, CODI forms faithful bridge states across latent-thought positions, while the final input follows a separate near-direct route; predictions arise through late fusion at the answer readout. As task depth and difficulty increase, however, CODI does not reliably sustain a full latent rollout: it either compresses computation into a partial late-intermediate pathway or, in harder regimes, loses the latent reasoning signature altogether. To explain this transition, we show theoretically that the task's algebraic structure controls its effective memory: compressible regimes support late-bottleneck reasoning, while incompressible regimes preserve full-history dependence and destabilize latent rollouts. Overall, our results characterize when CODI-style latent-CoT yields faithful iterative computation versus compressed or shortcut strategies.