# Do language models plan ahead for future tokens?

arXiv: https://arxiv.org/abs/2404.00859 · HTML: https://mlsanity.com/paper/2404.00859/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 36 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://mlsanity.com/paper/2401.10774/) (2024)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [Crowdsourcing Multiple Choice Science Questions](https://mlsanity.com/paper/1707.06209/) (2017)
- [Compressive Transformers for Long-Range Sequence Modelling](https://mlsanity.com/paper/1911.05507/) (2019)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [Blockwise Parallel Decoding for Deep Autoregressive Models](https://mlsanity.com/paper/1811.03115/) (2018)
- [Analysis Methods in Neural Language Processing: A Survey](https://mlsanity.com/paper/1812.08951/) (2018)
- [Understanding the Role of Individual Units in a Deep Neural Network](https://mlsanity.com/paper/2009.05041/) (2020)
- [Information-Theoretic Probing for Linguistic Structure](https://mlsanity.com/paper/2004.03061/) (2020)
- [Linearity of Relation Decoding in Transformer Language Models](https://mlsanity.com/paper/2308.09124/) (2023)
- [A Theory of Usable Information Under Computational Constraints](https://mlsanity.com/paper/2002.10689/) (2020)
- [Future Lens: Anticipating Subsequent Tokens from a Single Hidden State](https://mlsanity.com/paper/2311.04897/) (2023)
- [Optimal Approximation Rate of ReLU Networks in terms of Width and Depth](https://mlsanity.com/paper/2103.00502/) (2021)
- [KG^2: Learning to Reason Science Exam Questions with Contextual Knowledge Graph Embeddings](https://mlsanity.com/paper/1805.12393/) (2018)

## Built on by (most cited first, 30 of 49)

- [Scaling Diffusion Language Models via Adaptation from Autoregressive Models](https://mlsanity.com/paper/2410.17891/) (2024)
- [Let's Think Dot by Dot: Hidden Computation in Transformer Language Models](https://mlsanity.com/paper/2404.15758/) (2024)
- [Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization](https://mlsanity.com/paper/2405.15071/) (2024)
- [Large Language Models for Planning: A Comprehensive and Systematic Survey](https://mlsanity.com/paper/2505.19683/) (2025)
- [Activation Scaling for Steering and Interpreting Language Models](https://mlsanity.com/paper/2410.04962/) (2024)
- [How Far Are We From AGI: Are LLMs All We Need?](https://mlsanity.com/paper/2405.10313/) (2024)
- [Non-myopic Generation of Language Models for Reasoning and Planning](https://mlsanity.com/paper/2410.17195/) (2024)
- [LLM Probability Concentration: How Alignment Shrinks the Generative Horizon](https://mlsanity.com/paper/2506.17871/) (2025)
- [AdvPrefix: An Objective for Nuanced LLM Jailbreaks](https://mlsanity.com/paper/2412.10321/) (2024)
- [The Dual-Route Model of Induction](https://mlsanity.com/paper/2504.03022/) (2025)
- [Unlocking the Future: Exploring Look-Ahead Planning Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2406.16033/) (2024)
- [Emergent Response Planning in LLMs](https://mlsanity.com/paper/2502.06258/) (2025)
- [AdaEAGLE: Optimizing Speculative Decoding via Explicit Modeling of Adaptive Draft Structures](https://mlsanity.com/paper/2412.18910/) (2024)
- [Large Language Models Are Cross-Lingual Knowledge-Free Reasoners](https://mlsanity.com/paper/2406.16655/) (2024)
- [How Do LLMs Use Their Depth?](https://mlsanity.com/paper/2510.18871/) (2025)
- [L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Models](https://mlsanity.com/paper/2505.17505/) (2025)
- [Is the Reversal Curse a Binding Problem? Uncovering Limitations of Transformers from a Basic Generalization Failure](https://mlsanity.com/paper/2504.01928/) (2025)
- [On Reasoning Strength Planning in Large Reasoning Models](https://mlsanity.com/paper/2506.08390/) (2025)
- [Extracting Paragraphs from LLM Token Activations](https://mlsanity.com/paper/2409.06328/) (2024)
- [Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners](https://mlsanity.com/paper/2601.02996/) (2026)
- [Transforming Agency. On the mode of existence of Large Language Models](https://mlsanity.com/paper/2407.10735/) (2024)
- [What's the plan? Metrics for implicit planning in LLMs and their application to rhyme generation and question answering](https://mlsanity.com/paper/2601.20164/) (2026)
- [Catching rationalization in the act: detecting motivated reasoning before and after CoT via activation probing](https://mlsanity.com/paper/2603.17199/) (2026)
- [Latent Planning Emerges with Scale](https://mlsanity.com/paper/2604.12493/) (2026)
- [Looking beyond the next token](https://mlsanity.com/paper/2504.11336/) (2025)
- [Quantum Hamiltonian Embedding of Images for Data Reuploading Classifiers](https://mlsanity.com/paper/2407.14055/) (2024)
- [Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modeling](https://mlsanity.com/paper/2410.01440/) (2024)
- [Are Language Models Efficient Reasoners? A Perspective from Logic Programming](https://mlsanity.com/paper/2510.25626/) (2025)
- [Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More](https://mlsanity.com/paper/2503.10542/) (2025)
- [Modes of Sequence Models and Learning Coefficients](https://mlsanity.com/paper/2504.18048/) (2025)

## Abstract (arXiv)

Do transformers "think ahead" during inference at a given position? It is known transformers prepare information in the hidden states of the forward pass at time step t that is then used in future forward passes t+τ. We posit two explanations for this phenomenon: pre-caching, in which off-diagonal gradient terms present during training result in the model computing features at t irrelevant to the present inference task but useful for the future, and breadcrumbs, in which features most relevant to time step t are already the same as those that would most benefit inference at time t+τ. We test these hypotheses by training language models without propagating gradients to past timesteps, a scheme we formalize as myopic training. In a constructed synthetic data setting, we find clear evidence for pre-caching. In the autoregressive language modeling setting, our experiments are more suggestive of the breadcrumbs hypothesis, though pre-caching increases with model scale.