mlsanityData to 7 Oct 2026

LLM+P: Empowering Large Language Models with Optimal Planning Proficiency

arXiv 2304.11477 ↗First posted 22 Apr 2023cs.AI cs.ROcited by 496

Not on the map. Why?

Cited by 145 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) have demonstrated remarkable zero-shot generalization abilities: state-of-the-art chatbots can provide plausible answers to many common questions that arise in daily life. However, so far, LLMs cannot reliably solve long-horizon planning problems. By contrast, classical planners, once a problem is given in a formatted way, can use efficient search algorithms to quickly identify correct, or even optimal, plans. In an effort to get the best of both worlds, this paper introduces LLM+P, the first framework that incorporates the strengths of classical planners into LLMs. LLM+P takes in a natural language description of a planning problem, then returns a correct (or optimal) plan for solving that problem in natural language. LLM+P does so by first converting the language description into a file written in the planning domain definition language (PDDL), then leveraging classical planners to quickly find a solution, and then translating the found solution back into natural language. Along with LLM+P, we define a diverse set of different benchmark problems taken from common planning scenarios. Via a comprehensive set of experiments on these benchmark problems, we find that LLM+P is able to provide optimal solutions for most problems, while LLMs fail to provide even feasible plans for most problems.footnoteThe code and results are publicly available at https://github.com/Cranial-XIX/llm-pddl.git.

Key facts. Cited by 496 papers in the map (data through 7 Oct 2026). Builds on 34 of its 90 references that are in the map. First posted on arXiv 22 Apr 2023.

Builds on

34 of 90 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  11. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  12. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
22 more
  1. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  2. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  3. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  4. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  5. 2022PAL: Program-aided Language Modelscited by 635
  6. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  7. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  8. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 372
  9. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 296
  10. 2023Faithful Chain-of-Thought Reasoningcited by 285
  11. 2023Text2Motion: From Natural Language Instructions to Feasible Planscited by 223
  12. 2019Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Modelscited by 141
  13. 2023Task and Motion Planning with Large Language Models for Object Rearrangementcited by 140
  14. 2023Translating Natural Language to Planning Goals with Large-Language Modelscited by 132
  15. 2022Internet-augmented language models through few-shot prompting for open-domain question answeringcited by 131
  16. 2023On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)cited by 85
  17. 2022Housekeep: Tidying Virtual Households using Commonsense Reasoningcited by 57
  18. 2022Plansformer: Generating Symbolic Plans using Transformerscited by 41
  19. 2018Task Planning in Robotics: an Empirical Comparison of PDDL-based and ASP-based Systemscited by 40
  20. 2022Automaton-Based Representations of Task Knowledge from Generative Language Modelscited by 12
  21. 2022Robot Task Planning and Situation Handling in Open Worldscited by 10
  22. 2020Task-Motion Planning for Safe and Efficient Urban Drivingcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 76; 2024: 171; 2025: 141; 2026: 108.

496 papers.

Sort:

By area:

  1. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  2. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  3. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  4. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  5. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  6. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  7. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  8. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  9. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 514
  10. 2023Cognitive Architectures for Language Agentscited by 422
  11. 2024Understanding the planning of LLM agents: A surveycited by 414
  12. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 385
38 more
  1. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
  2. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
  3. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  4. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 308
  5. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 298
  6. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
  7. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  8. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  9. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 257
  10. 2023FireAct: Toward Language Agent Fine-tuningcited by 247
  11. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  12. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 226
  13. 2023Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planningcited by 204
  14. 2023Large Language Models as General Pattern Machinescited by 196
  15. 2023AdaPlanner: Adaptive Planning from Feedback with Language Modelscited by 193
  16. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 187
  17. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  18. 2024LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworkscited by 178
  19. 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 176
  20. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  21. 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Modelscited by 170
  22. 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
  23. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fastcited by 149
  24. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 148
  25. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 146
  26. 2023Generalized Planning in PDDL Domains with Pretrained Large Language Modelscited by 139
  27. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agentscited by 135
  28. 2023Theory of Mind for Multi-Agent Collaboration via Large Language Modelscited by 129
  29. 2024NATURAL PLAN: Benchmarking LLMs on Natural Language Planningcited by 125
  30. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  31. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thoughtcited by 118
  32. 2024Empowering Biomedical Discovery with AI Agentscited by 117
  33. 2023AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkerscited by 111
  34. 2024HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Modelcited by 101
  35. 2023SatLM: Satisfiability-Aided Language Models Using Declarative Promptingcited by 100
  36. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  37. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 96
  38. 2024Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planningcited by 95

50 most cited of 496 All 496 (JSON)