# Symmetry-Aware Transformer Training for Automated Planning

arXiv: https://arxiv.org/abs/2508.07743 · HTML: https://mlsanity.com/paper/2508.07743/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 11 Aug 2025.

## Builds on (most cited first, 19 of 19)

- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](https://mlsanity.com/paper/2304.11477/) (2023)
- [The Impact of Positional Encoding on Length Generalization in Transformers](https://mlsanity.com/paper/2305.19466/) (2023)
- [The LAMA Planner: Guiding Cost-Based Anytime Planning with Landmarks](https://mlsanity.com/paper/1401.3839/) (2014)
- [Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces](https://mlsanity.com/paper/2410.09918/) (2024)
- [ASNets: Deep Learning for Generalised Planning](https://mlsanity.com/paper/1908.01362/) (2019)
- [Plansformer: Generating Symbolic Plans using Transformers](https://mlsanity.com/paper/2212.08681/) (2022)
- [Learning General Optimal Policies with Graph Neural Networks: Expressive Power, Transparency, and Limits](https://mlsanity.com/paper/2109.10129/) (2021)
- [Thought of Search: Planning with Language Models Through The Lens of Efficiency](https://mlsanity.com/paper/2404.11833/) (2024)
- [Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges](https://mlsanity.com/paper/2502.01612/) (2025)
- [Learning Generalized Policies Without Supervision Using GNNs](https://mlsanity.com/paper/2205.06002/) (2022)
- [NL2Plan: Robust LLM-Driven Planning from Minimal Text Descriptions](https://mlsanity.com/paper/2405.04215/) (2024)
- [Return to Tradition: Learning Reliable Heuristics with Classical Machine Learning](https://mlsanity.com/paper/2403.16508/) (2024)
- [Planning in the Dark: LLM-Symbolic Planning Pipeline without Experts](https://mlsanity.com/paper/2409.15915/) (2024)
- [LLMs Can Plan Only If We Tell Them](https://mlsanity.com/paper/2501.13545/) (2025)
- [Learning More Expressive General Policies for Classical Planning Domains](https://mlsanity.com/paper/2403.11734/) (2024)

## Built on by (most cited first, 5 of 5)

- [On the Ability of Transformers to Verify Plans](https://mlsanity.com/paper/2603.19954/) (2026)
- [Self-Improvement for Fast, High-Quality Plan Generation](https://mlsanity.com/paper/2605.03625/) (2026)
- [LLM-Evolved Domain-Independent Heuristics for Symbolic AI Planning](https://mlsanity.com/paper/2605.29649/) (2026)
- [On Sample-Efficient Generalized Planning via Learned Transition Models](https://mlsanity.com/paper/2602.23148/) (2026)
- [Efficient Test-time Inference for Generative Planning Models with OCL Search](https://mlsanity.com/paper/2606.00618/) (2026)

## Abstract (arXiv)

While transformers excel in many settings, their application in the field of automated planning is limited. Prior work like PlanGPT, a state-of-the-art decoder-only transformer, struggles with extrapolation from easy to hard planning problems. This in turn stems from problem symmetries: planning tasks can be represented with arbitrary variable names that carry no meaning beyond being identifiers. This causes a combinatorial explosion of equivalent representations that pure transformers cannot efficiently learn from. We propose a novel contrastive learning objective to make transformers symmetry-aware and thereby compensate for their lack of inductive bias. Combining this with architectural improvements, we show that transformers can be efficiently trained for either plan-generation or heuristic-prediction. Our results across multiple planning domains demonstrate that our symmetry-aware training effectively and efficiently addresses the limitations of PlanGPT.