# LLMs as Planning Formalizers: A Survey for Leveraging Large Language Models to Construct Automated Planning Models

arXiv: https://arxiv.org/abs/2503.18971 · HTML: https://mlsanity.com/paper/2503.18971/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 70 of its 126 references that are in the map.
- First posted on arXiv 22 Mar 2025.

## Builds on (most cited first, 30 of 70)

- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](https://mlsanity.com/paper/2304.11477/) (2023)
- [Understanding the planning of LLM agents: A survey](https://mlsanity.com/paper/2402.02716/) (2024)
- [TravelPlanner: A Benchmark for Real-World Planning with Language Agents](https://mlsanity.com/paper/2402.01622/) (2024)
- [Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning](https://mlsanity.com/paper/2305.12295/) (2023)
- [PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change](https://mlsanity.com/paper/2206.10498/) (2022)
- [Faithful Chain-of-Thought Reasoning](https://mlsanity.com/paper/2301.13379/) (2023)
- [Text2Motion: From Natural Language Instructions to Feasible Plans](https://mlsanity.com/paper/2303.12153/) (2023)
- [Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning](https://mlsanity.com/paper/2305.14909/) (2023)
- [LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks](https://mlsanity.com/paper/2402.01817/) (2024)
- [Task and Motion Planning with Large Language Models for Object Rearrangement](https://mlsanity.com/paper/2303.06247/) (2023)
- [NATURAL PLAN: Benchmarking LLMs on Natural Language Planning](https://mlsanity.com/paper/2406.04520/) (2024)
- [AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkers](https://mlsanity.com/paper/2306.06531/) (2023)
- [Can Large Language Models Really Improve by Self-critiquing Their Own Plans?](https://mlsanity.com/paper/2310.08118/) (2023)
- [Chain of Thoughtlessness? An Analysis of CoT in Planning](https://mlsanity.com/paper/2405.04776/) (2024)
- [LLMs Still Can't Plan; Can LRMs? A Preliminary Evaluation of OpenAI's o1 on PlanBench](https://mlsanity.com/paper/2409.13373/) (2024)
- [GPT-4 Doesn't Know It's Wrong: An Analysis of Iterative Prompting for Reasoning Problems](https://mlsanity.com/paper/2310.12397/) (2023)
- [AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation](https://mlsanity.com/paper/2408.00764/) (2024)
- [Planning In Natural Language Improves LLM Search For Code Generation](https://mlsanity.com/paper/2409.03733/) (2024)
- [ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning](https://mlsanity.com/paper/2308.13724/) (2023)
- [Evaluating Cognitive Maps and Planning in Large Language Models with CogEval](https://mlsanity.com/paper/2309.15129/) (2023)
- [Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks](https://mlsanity.com/paper/2205.05718/) (2022)
- [Plansformer: Generating Symbolic Plans using Transformers](https://mlsanity.com/paper/2212.08681/) (2022)
- [On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)](https://mlsanity.com/paper/2401.02500/) (2024)
- [ACPBench: Reasoning about Action, Change, and Planning](https://mlsanity.com/paper/2410.05669/) (2024)
- [Planning Anything with Rigor: General-Purpose Zero-Shot Planning with LLM-based Formalized Programming](https://mlsanity.com/paper/2410.12112/) (2024)
- [Robust Planning with LLM-Modulo Framework: Case Study in Travel Planning](https://mlsanity.com/paper/2405.20625/) (2024)

## Built on by (most cited first, 30 of 38)

- [Web-Bench: A LLM Code Benchmark Based on Web Standards and Frameworks](https://mlsanity.com/paper/2505.07473/) (2025)
- [Large Language Models for Planning: A Comprehensive and Systematic Survey](https://mlsanity.com/paper/2505.19683/) (2025)
- [Classical Planning with LLM-Generated Heuristics: Challenging the State of the Art with Python Code](https://mlsanity.com/paper/2503.18809/) (2025)
- [Teaching LLMs to Plan: Logical Chain-of-Thought Instruction Tuning for Symbolic Planning](https://mlsanity.com/paper/2509.13351/) (2025)
- [Language Models For Generalised PDDL Planning: Synthesising Sound and Programmatic Policies](https://mlsanity.com/paper/2508.18507/) (2025)
- [PatchWorld: Gradient-Free Optimization of Executable World Models for Agent Environments](https://mlsanity.com/paper/2605.30880/) (2026)
- [Using large language models for embodied planning introduces systematic safety risks](https://mlsanity.com/paper/2604.18463/) (2026)
- [Unifying Inference-Time Planning Language Generation](https://mlsanity.com/paper/2505.14763/) (2025)
- [Evaluating the Efficacy of LLM-Based Reasoning for Multiobjective HPC Job Scheduling](https://mlsanity.com/paper/2506.02025/) (2025)
- [Vision Language Models Cannot Plan, but Can They Formalize?](https://mlsanity.com/paper/2509.21576/) (2025)
- [Language Model as Planner and Formalizer under Constraints](https://mlsanity.com/paper/2510.05486/) (2025)
- [Achieving Scalable Robot Autonomy via neurosymbolic planning using lightweight local LLM](https://mlsanity.com/paper/2505.08492/) (2025)
- [Toward Safe LLM Agents: A Survey of Specification, Verification, and Enforcement](https://mlsanity.com/paper/2608.14590/) (2026)
- [Learning Hierarchical Domain Models Through Environment-Grounded Interaction](https://mlsanity.com/paper/2505.13497/) (2025)
- [Bridging Engineering and AI Planning through Model-Based Knowledge Transformation for the Validation of Automated Production System Variants](https://mlsanity.com/paper/2509.12091/) (2025)
- [Agentic LLM Planning via Step-Wise PDDL Simulation: An Empirical Characterisation](https://mlsanity.com/paper/2603.06064/) (2026)
- [Language Models as Higher-Order Planning Formalizers](https://mlsanity.com/paper/2603.23844/) (2026)
- [LLM-Evolved Domain-Independent Heuristics for Symbolic AI Planning](https://mlsanity.com/paper/2605.29649/) (2026)
- [The Horizon Gap: Planning, Memory, Execution, Training, and Evaluation for Long-Horizon LLM Agents](https://mlsanity.com/paper/2608.06663/) (2026)
- [Make Planning Research Rigorous Again!](https://mlsanity.com/paper/2505.21674/) (2025)
- [Scaling Multi-Agent Epistemic Planning through GNN-Derived Heuristics](https://mlsanity.com/paper/2508.12840/) (2025)
- [Autonomous Assessment of Generalizability of AI Agent Capabilities](https://mlsanity.com/paper/2512.16733/) (2025)
- [Textual Planning with Explicit Latent Transitions](https://mlsanity.com/paper/2602.04557/) (2026)
- [Generative AI-assisted Participatory Modeling in Socio-Environmental Planning under Deep Uncertainty](https://mlsanity.com/paper/2603.17021/) (2026)
- [RAMP: Hybrid DRL for Online Learning of Numeric Action Models](https://mlsanity.com/paper/2604.08685/) (2026)
- [Property-Guided LLM Program Synthesis for Planning](https://mlsanity.com/paper/2605.16142/) (2026)
- [Robust Asynchronous Planning via Auto-Formalization](https://mlsanity.com/paper/2606.00981/) (2026)
- [From Capability Models to Automated Planning: An AAS-Native Approach for Automatic PDDL Generation](https://mlsanity.com/paper/2606.02167/) (2026)
- [QueryGraph: Reliable Multi-Tool Query Execution Planning via LLM-Based Graph Generation](https://mlsanity.com/paper/2606.08300/) (2026)
- [Neuro-Symbolic Agents for Regulated Process Automation: Challenges and Research Agenda](https://mlsanity.com/paper/2606.13405/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) excel in various natural language tasks but often struggle with long-horizon planning problems requiring structured reasoning. This limitation has drawn interest in integrating neuro-symbolic approaches within the Automated Planning (AP) and Natural Language Processing (NLP) communities. However, identifying optimal AP deployment frameworks can be daunting and introduces new challenges. This paper aims to provide a timely survey of the current research with an in-depth analysis, positioning LLMs as tools for formalizing and refining planning specifications to support reliable off-the-shelf AP planners. By systematically reviewing the current state of research, we highlight methodologies, and identify critical challenges and future directions, hoping to contribute to the joint research on NLP and Automated Planning.