# On the Generalization Gap in LLM Planning: Tests and Verifier-Reward RL

arXiv: https://arxiv.org/abs/2601.14456 · HTML: https://mlsanity.com/paper/2601.14456/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 29 references that are in the map.
- First posted on arXiv 20 Jan 2026.

## Builds on (most cited first, 22 of 22)

- [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://mlsanity.com/paper/2504.13837/) (2025)
- [Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models](https://mlsanity.com/paper/2305.04091/) (2023)
- [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](https://mlsanity.com/paper/2304.11477/) (2023)
- [LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models](https://mlsanity.com/paper/2212.04088/) (2022)
- [RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2504.20073/) (2025)
- [Absolute Zero: Reinforced Self-play Reasoning with Zero Data](https://mlsanity.com/paper/2505.03335/) (2025)
- [The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity](https://mlsanity.com/paper/2506.06941/) (2025)
- [Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learning](https://mlsanity.com/paper/2505.01441/) (2025)
- [Faithful Logical Reasoning via Symbolic Chain-of-Thought](https://mlsanity.com/paper/2405.18357/) (2024)
- [Reasoning with Sampling: Your Base Model is Smarter Than You Think](https://mlsanity.com/paper/2510.14901/) (2025)
- [LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contexts](https://mlsanity.com/paper/2510.19363/) (2025)
- [Neural-Symbolic Reasoning over Knowledge Graphs: A Survey from a Query Perspective](https://mlsanity.com/paper/2412.10390/) (2024)
- [Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions](https://mlsanity.com/paper/2502.12616/) (2025)
- [Teaching LLMs to Plan: Logical Chain-of-Thought Instruction Tuning for Symbolic Planning](https://mlsanity.com/paper/2509.13351/) (2025)
- [A Framework for Neurosymbolic Robot Action Planning using Large Language Models](https://mlsanity.com/paper/2303.00438/) (2023)
- [Planning without Search: Refining Frontier LLMs with Offline Goal-Conditioned RL](https://mlsanity.com/paper/2505.18098/) (2025)
- [Language Models can Infer Action Semantics for Symbolic Planners from Environment Feedback](https://mlsanity.com/paper/2406.02791/) (2024)
- [Can LLM-Reasoning Models Replace Classical Planning? A Benchmark Study](https://mlsanity.com/paper/2507.23589/) (2025)
- [Fine-Tune an SLM or Prompt an LLM? The Case of Generating Low-Code Workflows](https://mlsanity.com/paper/2505.24189/) (2025)
- [A Comparative Study of Neurosymbolic AI Approaches to Interpretable Logical Reasoning](https://mlsanity.com/paper/2508.03366/) (2025)
- [A Systematic Study of Large Language Models for Task and Motion Planning With PDDLStream](https://mlsanity.com/paper/2510.00182/) (2025)
- [Achieving Scalable Robot Autonomy via neurosymbolic planning using lightweight local LLM](https://mlsanity.com/paper/2505.08492/) (2025)

## Built on by (most cited first, 1 of 1)

- [Toward Secure and Reliable PDDL Formalization of Large Language Models with Planner-in-the-Loop Feedback](https://mlsanity.com/paper/2606.29700/) (2026)

## Abstract (arXiv)

Recent work shows that fine-tuned Large Language Models (LLMs) can achieve high valid plan rates on PDDL planning tasks. However, it remains unclear whether this reflects transferable planning competence or domain-specific memorization. In this work, we fine-tune a 1.7B-parameter LLM on 40,000 domain-problem-plan tuples from 10 IPC 2023 domains, and evaluate both in-domain and cross-domain generalization. While the model reaches 82.9% valid plan rate in in-domain conditions, it achieves 0% on two unseen domains. To analyze this failure, we introduce three diagnostic interventions, namely (i) instance-wise symbol anonymization, (ii) compact plan serialization, and (iii) verifier-reward fine-tuning using the VAL validator as a success-focused reinforcement signal. Symbol anonymization and compact serialization cause significant performance drops despite preserving plan semantics, thus revealing strong sensitivity to surface representations. Verifier-reward fine-tuning reaches performance saturation in half the supervised training epochs, but does not improve cross-domain generalization. For the explored configurations, in-domain performance plateaus around 80%, while cross-domain performance collapses, suggesting that our fine-tuned model relies heavily on domain-specific patterns rather than transferable planning competence in this setting. Our results highlight a persistent generalization gap in LLM-based planning and provide diagnostic tools for studying its causes.