mlsanityData to 14 Oct 2026

Toward Secure and Reliable PDDL Formalization of Large Language Models with Planner-in-the-Loop Feedback

arXiv 2606.29700 ↗First posted 29 Jun 2026cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Process reward models for reasoning

Abstract (arXiv)

Planning often requires symbolic specifications that are both executable and verifiable. For large language models deployed in autonomous or decision-support systems, failures in such formalization may lead to unverifiable decisions, execution failures, or unsafe downstream behavior. We present NL-PDDL-Bench, a multi-domain benchmark for natural-language-to-PDDL specification construction with planner-verified executability and controlled difficulty scaling by object count. We further propose a planner-in-the-loop framework that uses validator and planner diagnostics to revise non-executable specifications through localized edits. Building on this infrastructure, we develop a planner-grounded optimization recipe that combines parameter-efficient Low-Rank Adaptation supervised fine-tuning, offline planner-derived preference pairs for Direct Preference Optimization, and inference-time planner-in-the-loop repair, without requiring online planner calls during training. We also provide a unified evaluation suite for parseability, solvability, specification similarity, and outcome-aware plan-level consistency against planner references. Experiments on representative model families show substantial gains in planner success and plan-level agreement, with improved robustness under difficulty scaling and cross-domain variation. These results highlight the value of externally verifiable formalization for reliable deployment of LLMs in safety- or security-sensitive planning systems. Code and data are available at: https://github.com/ibasicplan/NL-PDDL-Bench

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Process reward models for reasoning, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 14 of its 24 references that are in the map. First posted on arXiv 29 Jun 2026.

Builds on

14 of 24 references.

References · 14

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  4. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  5. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 382
  6. 2023Generalized Planning in PDDL Domains with Pretrained Large Language Modelscited by 139
  7. 2023On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)cited by 85
  8. 2024ACPBench: Reasoning about Action, Change, and Planningcited by 39
  9. 2025LLMs as Planning Formalizers: A Survey for Leveraging Large Language Models to Construct Automated Planning Modelscited by 38
  10. 2024NL2Plan: Robust LLM-Driven Planning from Minimal Text Descriptionscited by 22
  11. 2024Leveraging Environment Interaction for Automated PDDL Translation and Planning with Large Language Modelscited by 17
  12. 2024Planning in the Dark: LLM-Symbolic Planning Pipeline without Expertscited by 16
2 more
  1. 2024Unlocking Large Language Model's Planning Capabilities with Maximum Diversity Fine-tuningcited by 6
  2. 2026On the Generalization Gap in LLM Planning: Tests and Verifier-Reward RLcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026PDDLCoder: Agentic PDDL Generation for LLM-Assisted Symbolic Planning
  2. 2026From Solver Feedback to Faithful Plans: Multi-Role Reinforcement Learning for Symbolic Planning
  3. 2025Generating Symbolic World Models via Test-time Scaling of Large Language Models
  4. 2025Language Models For Generalised PDDL Planning: Synthesising Sound and Programmatic Policies
  5. 2025An Extensive Evaluation of PDDL Capabilities in off-the-shelf LLMs
  6. 2026Language Models as Higher-Order Planning Formalizers
  7. 2023Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning
  8. 2026Towards Reliable and Robust LLM Planning: Symbolic Feedback-Driven Iterative Self-Refinement Framework