Where's the Plan? Locating Latent Planning in Language Models with Lightweight Mechanistic Interventions
On the map: Mechanistic interpretability → Steering and interpreting LLM reasoning
Abstract (arXiv)
We study planning site formation in language models -- where internal representations of structurally-constrained future tokens form during the forward pass, and whether they causally drive generation. Using rhyming-couplet completion as a clean test of forward-looking constraint, we apply two lightweight methods (linear probing and activation patching) across Qwen3, Gemma-3, and Llama-3 at more than ten scales. Probing shows that future-rhyme information is linearly decodable at the line boundary, with signal that strengthens with scale in all three families. Activation patching reveals that only Gemma-3-27B causally relies on this encoding, exhibiting a handoff in which the causal driver migrates from the rhyme word to the line boundary around layer 30. Every other model we test conditions on the rhyme word throughout generation, with near-zero causal effect at the line boundary despite strong probe signal. We localize the Gemma-3-27B handoff to five attention heads through two-stage path patching that recover ~90% of the rhyme-routing capacity at the newline.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Steering and interpreting LLM reasoning, one of 13 directions in Mechanistic interpretability. Builds on 19 of its 25 references that are in the map. First posted on arXiv 8 May 2026.
Builds on
19 of 25 references.
Closest ancestors · 1
Same direction, newest first
Other references · 18
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
- 2023Steering Language Models With Activation Engineeringcited by 922
- 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
- 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
- 2019Designing and Interpreting Probes with Control Taskscited by 640
- 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
- 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
- 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
- 2024Transcoders Find Interpretable LLM Feature Circuitscited by 200
6 more
- 2023Localizing Model Behavior with Path Patchingcited by 195
- 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Modelscited by 191
- 2021Acquisition of Chess Knowledge in AlphaZerocited by 123
- 2024Evidence of Learned Look-Ahead in a Chess-Playing Neural Networkcited by 34
- 2026Latent Planning Emerges with Scalecited by 6
- 2025ParaScopes: What do Language Models Activations Encode About Future Text?cited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Internal Planning in Language Models: Characterizing Horizon and Branch Awareness
- 2026Planning or Improvisation? Stress-Testing the Poetry Planning Site on Open Models and Open Cross-Layer Transcoders
- 2024ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models
- 2026The Depth Ceiling: On the Limits of Large Language Models in Discovering Latent Planning
- 2024How language models extrapolate outside the training data: A case study in Textualized Gridworld
- 2026When Steering Fails in Latent Reasoning: A Latent-to-Language Transition Gap
- 2026Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization
- 2025Looking beyond the next token