mlsanityData to 13 Oct 2026

From Solver Feedback to Faithful Plans: Multi-Role Reinforcement Learning for Symbolic Planning

arXiv 2608.21897 ↗First posted 22 Aug 2026cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Reliable planning requires converting natural-language instructions into executable symbolic specifications, yet large language models remain brittle without costly PDDL annotations and may exploit solver success in semantically unfaithful ways. We study how to learn faithful natural-language-to-PDDL formalization using only solver feedback, without human-written demonstrations. We propose a solvergrounded multi-role reinforcement learning framework where a single language model acts as an Actor, Judge, and Editor for generation, verification, and repair. The Actor proposes PDDL specifications, the Judge provides a solver-calibrated quality signal, and the Editor performs bounded diagnostic-conditioned refinement. On PlanBench, our method improves average success from 35.5% for LLM+P to 70.8%, achieves 66.3% faithful success, and reduces semantic drift to 6.4%. These results show that organizing solver feedback into generation, verification, and repair roles enables more scalable and faithful annotation-free symbolic planning

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 25 of its 31 references that are in the map. First posted on arXiv 22 Aug 2026.

Builds on

25 of 31 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,320
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,091
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,247
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,156
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,181
  7. 2024A Survey on LLM-as-a-Judgecited by 1,399
  8. 2023Reasoning with Language Model is Planning with World Modelcited by 851
  9. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 483
  10. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 377
  11. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 335
  12. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 333
13 more
  1. 2023AdaPlanner: Adaptive Planning from Feedback with Language Modelscited by 193
  2. 2023Generalized Planning in PDDL Domains with Pretrained Large Language Modelscited by 139
  3. 2023Translating Natural Language to Planning Goals with Large-Language Modelscited by 133
  4. 2024Chain of Thoughtlessness? An Analysis of CoT in Planningcited by 102
  5. 2024LLMs Still Can't Plan; Can LRMs? A Preliminary Evaluation of OpenAI's o1 on PlanBenchcited by 94
  6. 2023ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planningcited by 77
  7. 2025Multi-Agent Evolve: LLM Self-Improve through Co-evolutioncited by 59
  8. 2024Planetarium: A Rigorous Benchmark for Translating Text to Structured Planning Languagescited by 33
  9. 2024On the Limit of Language Models as Planning Formalizerscited by 18
  10. 2024Leveraging Environment Interaction for Automated PDDL Translation and Planning with Large Language Modelscited by 17
  11. 2024PROC2PDDL: Open-Domain Planning Representations from Textscited by 15
  12. 2025Teaching LLMs to Plan: Logical Chain-of-Thought Instruction Tuning for Symbolic Planningcited by 15
  13. 2025CEC-Zero: Zero-Supervision Character Error Correction with Self-Generated Rewardscited by 10

Built on by

Not cited yet.