mlsanityData to 14 Oct 2026

Learning to Plan for Language Modeling from Unlabeled Data

arXiv 2404.00614 ↗First posted 31 Mar 2024cs.CL cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

By training to predict the next token in an unlabeled corpus, large language models learn to perform many tasks without any labeled data. However, their next-token-prediction objective arguably limits their performance in scenarios that require planning, such as writing a coherent article. In this paper, we train a module for planning the future writing process via a self-supervised learning objective. Given the textual context, this planning module learns to predict future abstract writing actions, which correspond to centroids in a clustered text embedding space. By conditioning on these actions, our model extends the successful language model formula to more abstract planning in an unsupervised way. Empirically, we demonstrate that our method improves language modeling performance in general, particularly with respect to the text structure. Because our framework uses a planner module that is unsupervised and external to the language model, new planner modules can be trained at large scale and easily be shared with the community.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 56 of its 80 references that are in the map. First posted on arXiv 31 Mar 2024.

Builds on

56 of 80 references.

References · 56

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  10. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  11. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  12. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
38 more
  1. 2016Pointer Sentinel Mixture Modelscited by 3,096
  2. 2022Emergent Abilities of Large Language Modelscited by 2,512
  3. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  4. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  5. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
  6. 2015Pointer Networkscited by 895
  7. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  8. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
  9. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  10. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  11. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  12. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  13. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 483
  14. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  15. 2023Benchmarking Large Language Models for News Summarizationcited by 445
  16. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
  17. 2023Language Models can Solve Computer Taskscited by 372
  18. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 312
  19. 2023Large Language Model Guided Tree-of-Thoughtcited by 221
  20. 2023ART: Automatic multi-step reasoning and tool-use for large language modelscited by 197
  21. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 195
  22. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 164
  23. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 160
  24. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  25. 2023Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Frameworkcited by 139
  26. 2023Controlled Decoding from Language Modelscited by 135
  27. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 129
  28. 2021A Recipe For Arbitrary Text Style Transfer with Large Language Modelscited by 107
  29. 2018Document-Level Neural Machine Translation with Hierarchical Attention Networkscited by 85
  30. 2021Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoningcited by 85
  31. 2023Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Modelcited by 79
  32. 2023PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedbackcited by 69
  33. 2023PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documentscited by 61
  34. 2023Guiding Language Model Reasoning with Planning Tokenscited by 58
  35. 2023RCOT: Detecting and Rectifying Factual Inconsistency in Reasoning by Reversing Chain-of-Thoughtcited by 47
  36. 2019Topic-Guided Variational Autoencoders for Text Generationcited by 44
  37. 2021Efficient (Soft) Q-Learning for Text Generation with Limited Good Datacited by 37
  38. 2024Algorithmic progress in language modelscited by 31

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 6; 2026: 1.

12 papers.

  1. 2024Large Concept Models: Language Modeling in a Sentence Representation Spacecited by 89
  2. 2024Unlocking Reasoning Potential in Large Langauge Models by Scaling Code-form Planningcited by 25
  3. 2025Emergent Response Planning in LLMscited by 19
  4. 2025a1: Steep Test-time Scaling Law via Environment Augmented Generationcited by 13
  5. 2025Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codescited by 7
  6. 2024RATIONALYST: Mining Implicit Rationales for Process Supervision of Reasoningcited by 6
  7. 2025Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanismcited by 4
  8. 2026LoopMTP: A looped transformer guided by latent multi-token predictioncited by 3
  9. 2024Learning to Plan Long-Term for Language Modelingcited by 1
  10. 2025Temporal horizons in forecasting: a performance-learnability trade-offcited by 1
  11. 2025Alternatives To Next Token Prediction In Text Generation -- A Surveycited by 1
  12. 2024End-to-end Planner Training for Language Modelingcited by 0