mlsanityData to 7 Oct 2026

LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks

arXiv 2402.01817 ↗First posted 2 Feb 2024cs.AI cs.LGcited by 178

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

There is considerable confusion about the role of Large Language Models (LLMs) in planning and reasoning tasks. On one side are over-optimistic claims that LLMs can indeed do these tasks with just the right prompting or self-verification strategies. On the other side are perhaps over-pessimistic claims that all that LLMs are good for in planning/reasoning tasks are as mere translators of the problem specification from one syntactic format to another, and ship the problem off to external symbolic solvers. In this position paper, we take the view that both these extremes are misguided. We argue that auto-regressive LLMs cannot, by themselves, do planning or self-verification (which is after all a form of reasoning), and shed some light on the reasons for misunderstandings in the literature. We will also argue that LLMs should be viewed as universal approximate knowledge sources that have much more meaningful roles to play in planning/reasoning tasks beyond simple front-end/back-end format translators. We present a vision of bf LLM-Modulo Frameworks that combine the strengths of LLMs with external model-based verifiers in a tighter bi-directional interaction regime. We will show how the models driving the external verifiers themselves can be acquired with the help of LLMs. We will also argue that rather than simply pipelining LLMs and symbolic components, this LLM-Modulo Framework provides a better neuro-symbolic approach that offers tighter integration between LLMs and symbolic components, and allows extending the scope of model-based planning/reasoning regimes towards more flexible knowledge, problem and preference specifications.

Key facts. Cited by 178 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 34 of its 52 references that are in the map. First posted on arXiv 2 Feb 2024.

Builds on

34 of 52 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023On the Planning Abilities of Large Language Models : A Critical Investigation25 May
  2. 2023Reasoning with Language Model is Planning with World Model24 May
  3. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change21 Jun

Other references · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  8. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  9. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  10. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  11. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  12. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
19 more
  1. 2022Large Language Models Can Self-Improvecited by 656
  2. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  3. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  4. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  5. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
  6. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 304
  7. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 288
  8. 2023Reward Design with Language Modelscited by 257
  9. 2023Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planningcited by 204
  10. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
  11. 2023CodePlan: Repository-level Coding using LLMs and Planningcited by 139
  12. 2023Translating Natural Language to Planning Goals with Large-Language Modelscited by 132
  13. 2023Can Large Language Models Really Improve by Self-critiquing Their Own Plans?cited by 107
  14. 2023GPT-4 Doesn't Know It's Wrong: An Analysis of Iterative Prompting for Reasoning Problemscited by 88
  15. 2023SayNav: Grounding Large Language Models for Dynamic Planning to Navigation in New Environmentscited by 53
  16. 2023Understanding the Capabilities of Large Language Models for Automated Planningcited by 34
  17. 2021GPT3-to-plan: Extracting plans from text using GPT-3cited by 26
  18. 2022Leveraging Approximate Symbolic Models for Reinforcement Learning via Skill Diversitycited by 22
  19. 2024Theory of Mind abilities of Large Language Models in Human-Robot Interaction : An Illusion?cited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 76; 2025: 64; 2026: 38.

178 papers.

Sort:

By area:

  1. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 306
  2. 2024AI Agents That Mattercited by 199
  3. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 141
  4. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 140
  5. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  6. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  7. 2024Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worsecited by 107
  8. 2025Agentic Large Language Models, a surveycited by 105
  9. 2024Best Practices and Lessons Learned on Synthetic Datacited by 104
  10. 2025Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Designcited by 100
  11. 2024LLMs Still Can't Plan; Can LRMs? A Preliminary Evaluation of OpenAI's o1 on PlanBenchcited by 93
  12. 2024Can Large Language Models Reason and Plan?cited by 82
38 more
  1. 2025PlanGenLLMs: A Modern Survey of LLM Planning Capabilitiescited by 82
  2. 2024AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generationcited by 80
  3. 2024Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systemscited by 79
  4. 2024Nicer Than Humans: How do Large Language Models Behave in the Prisoner's Dilemma?cited by 62
  5. 2025The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMscited by 62
  6. 2025Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistantcited by 54
  7. 2024The Limits of Inference Scaling Through Resamplingcited by 51
  8. 2024RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulationcited by 48
  9. 2024Large Language Models Can Solve Real-World Planning Rigorously with Formal Verification Toolscited by 47
  10. 2024VSP: Assessing the dual challenges of perception and reasoning in spatial planning tasks for VLMscited by 47
  11. 2025LLMs as Planning Formalizers: A Survey for Leveraging Large Language Models to Construct Automated Planning Modelscited by 38
  12. 2024System-1.x: Learning to Balance Fast and Slow Planning with Language Modelscited by 35
  13. 2024Planning Anything with Rigor: General-Purpose Zero-Shot Planning with LLM-based Formalized Programmingcited by 35
  14. 2024Data-driven Discovery with Large Generative Modelscited by 33
  15. 2024Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examplescited by 29
  16. 2024Multimodal Large Language Models for Inverse Molecular Design with Retrosynthetic Planningcited by 29
  17. 2024Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguitycited by 26
  18. 2024Steering Large Language Models between Code Execution and Textual Reasoningcited by 24
  19. 2024Task Success is not Enough: Investigating the Use of Video-Language Models as Behavior Critics for Catching Undesirable Agent Behaviorscited by 23
  20. 2025Foundation Models for Spatio-Temporal Data Science: A Tutorial and Surveycited by 23
  21. 2024VLM-Auto: VLM-based Autonomous Driving Assistant with Human-like Behavior and Understanding for Complex Road Scenescited by 22
  22. 2024LLM Uncertainty Quantification through Directional Entailment Graph and Claim Level Response Augmentationcited by 22
  23. 2024LLM-based Conversational AI Therapist for Daily Functioning Screening and Psychotherapeutic Intervention via Everyday Smart Devicescited by 21
  24. 2024Trust the PRoC3S: Solving Long-Horizon Robotics Problems with LLMs and Constraint Satisfactioncited by 19
  25. 2024Exploring and Benchmarking the Planning Capabilities of Large Language Modelscited by 19
  26. 2024Transformers Struggle to Learn to Searchcited by 19
  27. 2025VeriPlan: Integrating Formal Verification and LLMs into End-User Planningcited by 19
  28. 2024Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2)cited by 17
  29. 2024On the Brittle Foundations of ReAct Prompting for Agentic Large Language Modelscited by 17
  30. 2024TRIP-PAL: Travel Planning with Guarantees by Combining Large Language Models and Automated Plannerscited by 17
  31. 2024Planning in Strawberry Fields: Evaluating and Improving the Planning and Scheduling Capabilities of LRM o1cited by 17
  32. 2024On the Limit of Language Models as Planning Formalizerscited by 17
  33. 2024Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Searchcited by 16
  34. 2024Can LLMs Reliably Simulate Human Learner Actions? A Simulation Authoring Framework for Open-Ended Learning Environmentscited by 16
  35. 2025Evaluating the Goal-Directedness of Large Language Modelscited by 15
  36. 2025Learning Compositional Behaviors from Demonstration and Languagecited by 15
  37. 2024RLSF: Fine-tuning LLMs via Symbolic Feedbackcited by 13
  38. 2024A Survey on Human-Centric LLMscited by 13

50 most cited of 178 All 178 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Can LLMs Fix Issues with Reasoning Models? Towards More Likely Models for AI Planning
  2. 2023On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)
  3. 2025Bridging the Reasoning Gap: Small LLMs Can Plan with Generalised Strategies
  4. 2026Language Models as Higher-Order Planning Formalizers
  5. 2024Chasing Progress, Not Perfection: Revisiting Strategies for End-to-End LLM Plan Generation
  6. 2024Robust Planning with LLM-Modulo Framework: Case Study in Travel Planning
  7. 2024On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks
  8. 2025LLMs Can Plan Only If We Tell Them