mlsanityData to 7 Oct 2026

Automating the Generation of Prompts for LLM-based Action Choice in PDDL Planning

arXiv 2311.09830 ↗First posted 16 Nov 2023cs.AI cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have revolutionized a large variety of NLP tasks. An active debate is to what extent they can do reasoning and planning. Prior work has assessed the latter in the specific context of PDDL planning, based on manually converting three PDDL domains into natural language (NL) prompts. Here we automate this conversion step, showing how to leverage an LLM to automatically generate NL prompts from PDDL input. Our automatically generated NL prompts result in similar LLM-planning performance as the previous manually generated ones. Beyond this, the automation enables us to run much larger experiments, providing for the first time a broad evaluation of LLM planning performance in PDDL. Our NL prompts yield better performance than PDDL prompts and simple template-based NL prompts. Compared to symbolic planners, LLM planning lags far behind; but in some domains, our best LLM configuration scales up further than A^star using LM-cut.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 14 of its 27 references that are in the map. First posted on arXiv 16 Nov 2023.

Builds on

14 of 27 references.

References · 14

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  3. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  4. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  5. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  6. 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
  7. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  8. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 372
  9. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 308
  10. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 298
  11. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 288
  12. 2023ADaPT: As-Needed Decomposition and Planning with Language Modelscited by 172
2 more
  1. 2023SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Taskscited by 166
  2. 2020Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Modelscited by 65

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 11; 2026: 5.

24 papers.

Sort:

By area:

  1. 2024ACPBench: Reasoning about Action, Change, and Planningcited by 37
  2. 2024Planning Anything with Rigor: General-Purpose Zero-Shot Planning with LLM-based Formalized Programmingcited by 35
  3. 2025Designing Domain-Specific Agents via Hierarchical Task Abstraction Mechanismcited by 16
  4. 2024PROC2PDDL: Open-Domain Planning Representations from Textscited by 15
  5. 2024PDDLEGO: Iterative Planning in Textual Environmentscited by 14
  6. 2024GameTraversalBenchmark: Evaluating Planning Abilities Of Large Language Models Through Traversing 2D Game Mapscited by 8
  7. 2025Can LLM-Reasoning Models Replace Classical Planning? A Benchmark Studycited by 8
  8. 2025Seemingly Simple Planning Problems are Computationally Challenging: The Countdown Gamecited by 7
  9. 2025Improved Generalized Planning with LLMs through Strategy Refinement and Reflectioncited by 7
  10. 2024What's the Plan? Evaluating and Developing Planning-Aware Techniques for Language Modelscited by 4
  11. 2025Doc2Agent: Scalable Generation of Tool-Using Agents from API Documentationcited by 4
  12. 2024PDDLFuse: A Tool for Generating Diverse Planning Domainscited by 3
12 more
  1. 2025Unifying Inference-Time Planning Language Generationcited by 3
  2. 2025Language Model as Planner and Formalizer under Constraintscited by 3
  3. 2024A Survey on Complex Tasks for Goal-Directed Interactive Agentscited by 2
  4. 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agentscited by 2
  5. 2025Evaluating the Process Modeling Abilities of Large Language Models -- Preliminary Foundations and Resultscited by 1
  6. 2025Iterative Formalization and Planning in Partially Observable Environmentscited by 1
  7. 2026On the Ability of Transformers to Verify Planscited by 1
  8. 2026Towards Reliable and Robust LLM Planning: Symbolic Feedback-Driven Iterative Self-Refinement Frameworkcited by 1
  9. 2026What We Talk About When We Talk About LLM Planning: Evidence for Two Distinct Planning Abilitiescited by 1
  10. 2025Make Planning Research Rigorous Again!cited by 0
  11. 2026Grounded Evaluation and Repair for NL-to-PDDL Problem Generationcited by 0
  12. 2026Verifiable Hidden Dynamics Play: Generating Agentic RL Environments from Solved Mechanismscited by 0