# LLMs Can Plan Only If We Tell Them

arXiv: https://arxiv.org/abs/2501.13545 · HTML: https://mlsanity.com/paper/2501.13545/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jan 2025.

## Built on by (most cited first, 15 of 15)

- [LLMs as Planning Formalizers: A Survey for Leveraging Large Language Models to Construct Automated Planning Models](https://mlsanity.com/paper/2503.18971/) (2025)
- [Classical Planning with LLM-Generated Heuristics: Challenging the State of the Art with Python Code](https://mlsanity.com/paper/2503.18809/) (2025)
- [CMCTS: A Constrained Monte Carlo Tree Search Framework for Mathematical Reasoning in Large Language Model](https://mlsanity.com/paper/2502.11169/) (2025)
- [Reason from Future: Reverse Thought Chain Enhances LLM Reasoning](https://mlsanity.com/paper/2506.03673/) (2025)
- [Symmetry-Aware Transformer Training for Automated Planning](https://mlsanity.com/paper/2508.07743/) (2025)
- [Backtracking for Safety](https://mlsanity.com/paper/2503.08919/) (2025)
- [MINDSTORES: Memory-Informed Neural Decision Synthesis for Task-Oriented Reinforcement in Embodied Systems](https://mlsanity.com/paper/2501.19318/) (2025)
- [Rethinking the Unsolvable: When In-Context Search Meets Test-Time Scaling](https://mlsanity.com/paper/2505.22290/) (2025)
- [Can LLMs Generate Good Stories? Insights and Challenges from a Narrative Planning Perspective](https://mlsanity.com/paper/2506.10161/) (2025)
- [Extracting Search Trees from LLM Reasoning Traces Reveals Myopic Planning](https://mlsanity.com/paper/2605.06840/) (2026)
- [Internal Planning in Language Models: Characterizing Horizon and Branch Awareness](https://mlsanity.com/paper/2509.25260/) (2025)
- [Language Models as Higher-Order Planning Formalizers](https://mlsanity.com/paper/2603.23844/) (2026)
- [LLMs for Resource Allocation: A Participatory Budgeting Approach to Inferring Preferences](https://mlsanity.com/paper/2508.06060/) (2025)
- [Reinforcement Learning with Backtracking Feedback](https://mlsanity.com/paper/2602.08377/) (2026)
- [Planning with Transformers: Chain of Computation and Structured Context Windows](https://mlsanity.com/paper/2607.17710/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated significant capabilities in natural language processing and reasoning, yet their effectiveness in autonomous planning has been under debate. While existing studies have utilized LLMs with external feedback mechanisms or in controlled environments for planning, these approaches often involve substantial computational and development resources due to the requirement for careful design and iterative backprompting. Moreover, even the most advanced LLMs like GPT-4 struggle to match human performance on standard planning benchmarks, such as the Blocksworld, without additional support. This paper investigates whether LLMs can independently generate long-horizon plans that rival human baselines. Our novel enhancements to Algorithm-of-Thoughts (AoT), which we dub AoT+, help achieve state-of-the-art results in planning benchmarks out-competing prior methods and human baselines all autonomously.