# Learning to Plan for Language Modeling from Unlabeled Data arXiv: https://arxiv.org/abs/2404.00614 ยท HTML: https://mlsanity.com/paper/2404.00614/ ## Key facts - Cited by 12 papers in the map (data through 14 Oct 2026). - Builds on 56 of its 80 references that are in the map. - First posted on arXiv 31 Mar 2024. ## Builds on (most cited first, 30 of 56) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019) - [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022) - [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021) - [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020) - [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019) - [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020) - [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022) - [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022) - [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023) - [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023) - [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016) - [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022) - [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020) - [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019) - [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020) - [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015) - [Reasoning with Language Model is Planning with World Model](https://mlsanity.com/paper/2305.14992/) (2023) - [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016) - [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019) - [LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention](https://mlsanity.com/paper/2303.16199/) (2023) - [OLMo: Accelerating the Science of Language Models](https://mlsanity.com/paper/2402.00838/) (2024) - [Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models](https://mlsanity.com/paper/2305.04091/) (2023) - [Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research](https://mlsanity.com/paper/2402.00159/) (2024) - [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021) - [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023) - [FUDGE: Controlled Text Generation With Future Discriminators](https://mlsanity.com/paper/2104.05218/) (2021) - [Language Models can Solve Computer Tasks](https://mlsanity.com/paper/2303.17491/) (2023) - [On the Planning Abilities of Large Language Models : A Critical Investigation](https://mlsanity.com/paper/2305.15771/) (2023) ## Built on by (most cited first, 12 of 12) - [Large Concept Models: Language Modeling in a Sentence Representation Space](https://mlsanity.com/paper/2412.08821/) (2024) - [Unlocking Reasoning Potential in Large Langauge Models by Scaling Code-form Planning](https://mlsanity.com/paper/2409.12452/) (2024) - [Emergent Response Planning in LLMs](https://mlsanity.com/paper/2502.06258/) (2025) - [a1: Steep Test-time Scaling Law via Environment Augmented Generation](https://mlsanity.com/paper/2504.14597/) (2025) - [Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes](https://mlsanity.com/paper/2505.04993/) (2025) - [RATIONALYST: Mining Implicit Rationales for Process Supervision of Reasoning](https://mlsanity.com/paper/2410.01044/) (2024) - [Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism](https://mlsanity.com/paper/2510.13170/) (2025) - [LoopMTP: A looped transformer guided by latent multi-token prediction](https://mlsanity.com/paper/2608.03624/) (2026) - [Learning to Plan Long-Term for Language Modeling](https://mlsanity.com/paper/2409.00070/) (2024) - [Temporal horizons in forecasting: a performance-learnability trade-off](https://mlsanity.com/paper/2506.03889/) (2025) - [Alternatives To Next Token Prediction In Text Generation -- A Survey](https://mlsanity.com/paper/2509.24435/) (2025) - [End-to-end Planner Training for Language Modeling](https://mlsanity.com/paper/2410.12492/) (2024) ## Abstract (arXiv) By training to predict the next token in an unlabeled corpus, large language models learn to perform many tasks without any labeled data. However, their next-token-prediction objective arguably limits their performance in scenarios that require planning, such as writing a coherent article. In this paper, we train a module for planning the future writing process via a self-supervised learning objective. Given the textual context, this planning module learns to predict future abstract writing actions, which correspond to centroids in a clustered text embedding space. By conditioning on these actions, our model extends the successful language model formula to more abstract planning in an unsupervised way. Empirically, we demonstrate that our method improves language modeling performance in general, particularly with respect to the text structure. Because our framework uses a planner module that is unsupervised and external to the language model, new planner modules can be trained at large scale and easily be shared with the community.