# Latent Planning Emerges with Scale

arXiv: https://arxiv.org/abs/2604.12493 · HTML: https://mlsanity.com/paper/2604.12493/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 43 references that are in the map.
- First posted on arXiv 14 Apr 2026.

## Builds on (most cited first, 29 of 29)

- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [TinyStories: How Small Can Language Models Be and Still Speak Coherent English?](https://mlsanity.com/paper/2305.07759/) (2023)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety](https://mlsanity.com/paper/2507.11473/) (2025)
- [Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models](https://mlsanity.com/paper/2401.06102/) (2024)
- [How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model](https://mlsanity.com/paper/2305.00586/) (2023)
- [Transcoders Find Interpretable LLM Feature Circuits](https://mlsanity.com/paper/2406.11944/) (2024)
- [The pitfalls of next-token prediction](https://mlsanity.com/paper/2403.06963/) (2024)
- [Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?](https://mlsanity.com/paper/2005.00719/) (2020)
- [Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models](https://mlsanity.com/paper/2106.06087/) (2021)
- [Future Lens: Anticipating Subsequent Tokens from a Single Hidden State](https://mlsanity.com/paper/2311.04897/) (2023)
- [Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information](https://mlsanity.com/paper/1808.08079/) (2018)
- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)
- [Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction](https://mlsanity.com/paper/2105.06965/) (2021)
- [Probing for the Usage of Grammatical Number](https://mlsanity.com/paper/2204.08831/) (2022)
- [Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languages](https://mlsanity.com/paper/2501.06346/) (2025)
- [Do language models plan ahead for future tokens?](https://mlsanity.com/paper/2404.00859/) (2024)
- [Language Model Circuits Are Sparse in the Neuron Basis](https://mlsanity.com/paper/2601.22594/) (2026)
- [Emergent Response Planning in LLMs](https://mlsanity.com/paper/2502.06258/) (2025)
- [Evaluating Diversity in Automatic Poetry Generation](https://mlsanity.com/paper/2406.15267/) (2024)
- [Bits of Grass: Does GPT already know how to write like Whitman?](https://mlsanity.com/paper/2305.11064/) (2023)
- [PhonologyBench: Evaluating Phonological Skills of Large Language Models](https://mlsanity.com/paper/2404.02456/) (2024)
- [What's the plan? Metrics for implicit planning in LLMs and their application to rhyme generation and question answering](https://mlsanity.com/paper/2601.20164/) (2026)
- [Incremental Sentence Processing Mechanisms in Autoregressive Transformer Language Models](https://mlsanity.com/paper/2412.05353/) (2024)

## Built on by (most cited first, 6 of 6)

- [Where is the Mind? Persona Vectors and LLM Individuation](https://mlsanity.com/paper/2604.17031/) (2026)
- [What Happens Inside Agent Memory? Circuit Analysis from Emergence to Diagnosis](https://mlsanity.com/paper/2605.03354/) (2026)
- [What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformers](https://mlsanity.com/paper/2604.15010/) (2026)
- [Where's the Plan? Locating Latent Planning in Language Models with Lightweight Mechanistic Interventions](https://mlsanity.com/paper/2605.07984/) (2026)
- [How Do Agentic LLMs Decide to Call Tools? A Tool-Call Vector Shaped by Suppression](https://mlsanity.com/paper/2610.09624/) (2026)
- [Narrative Lookahead in LLMs: Event-Level Decodability, Predictive Advantage, and Steering](https://mlsanity.com/paper/2610.18121/) (2026)

## Abstract (arXiv)

LLMs can perform seemingly planning-intensive tasks, like writing coherent stories or functioning code, without explicitly verbalizing a plan; however, the extent to which they implicitly plan is unknown. In this paper, we define latent planning as occurring when LLMs possess internal planning representations that (1) cause the generation of a specific future token or concept, and (2) shape preceding context to license said future token or concept. We study the Qwen-3 family (0.6B-14B) on simple planning tasks, finding that latent planning ability increases with scale. Models that plan possess features that represent a planned-for word like "accountant", and cause them to output "an" rather than "a"; moreover, even the less-successful Qwen-3 4B-8B have nascent planning mechanisms. On the more complex task of completing rhyming couplets, we find that models often identify a rhyme ahead of time, but even large models seldom plan far ahead. However, we can elicit some planning that increases with scale when steering models towards planned words in prose. In sum, we offer a framework for measuring planning and mechanistic evidence of how models' planning abilities grow with scale.