LLM-Only PDDL Domain Repair with Open-Weight Models
Not on the map. Why?
Abstract (arXiv)
AI planning is concerned with finding a sequence of actions that achieves a specified goal. It relies on explicit models of the world, commonly represented in the Planning Domain Definition Language (PDDL). An active line of research investigates how errors in such models can be detected and repaired. For example, users may provide positive test plans that are solutions, and negative test plans that fail during execution. Automated repair methods then modify the PDDL model to satisfy these constraints. In this paper, we evaluate the ability of recent open-weight large language models to perform this repair task using an LLM-only approach. Our experiments show that the symbolic baseline achieves an F1 score of .49, while the best-performing LLM reaches .87 with high reasoning effort, an absolute improvement of .38. However, that setting has a mean test pass rate of only .82, falling to .06 on the Thoughtful domain; even the best setting that includes the test traces reaches only .92. Thus, current open-weight models cannot guarantee satisfaction of the test constraints required for reliable automated model repair.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 4 of its 11 references that are in the map. First posted on arXiv 18 Aug 2026.
Builds on
4 of 11 references.
References · 4
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
- 2025LLMs as Planning Formalizers: A Survey for Leveraging Large Language Models to Construct Automated Planning Modelscited by 38
- 2024Planning in the Dark: LLM-Symbolic Planning Pipeline without Expertscited by 16
- 2024Chasing Progress, Not Perfection: Revisiting Strategies for End-to-End LLM Plan Generationcited by 11
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Testing and Understanding Erroneous Planning in LLM Agents through Synthesized User Inputs
- 2025An Extensive Evaluation of PDDL Capabilities in off-the-shelf LLMs
- 2026PDDLCoder: Agentic PDDL Generation for LLM-Assisted Symbolic Planning
- 2025Boosting Open-Source LLMs for Program Repair via Reasoning Transfer and LLM-Guided Reinforcement Learning
- 2025Can LLM-Reasoning Models Replace Classical Planning? A Benchmark Study
- 2023Can LLMs Fix Issues with Reasoning Models? Towards More Likely Models for AI Planning
- 2024Leveraging Environment Interaction for Automated PDDL Translation and Planning with Large Language Models
- 2023Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning