mlsanityData to 14 Oct 2026

Symmetry-Aware Transformer Training for Automated Planning

arXiv 2508.07743 ↗First posted 11 Aug 2025cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

While transformers excel in many settings, their application in the field of automated planning is limited. Prior work like PlanGPT, a state-of-the-art decoder-only transformer, struggles with extrapolation from easy to hard planning problems. This in turn stems from problem symmetries: planning tasks can be represented with arbitrary variable names that carry no meaning beyond being identifiers. This causes a combinatorial explosion of equivalent representations that pure transformers cannot efficiently learn from. We propose a novel contrastive learning objective to make transformers symmetry-aware and thereby compensate for their lack of inductive bias. Combining this with architectural improvements, we show that transformers can be efficiently trained for either plan-generation or heuristic-prediction. Our results across multiple planning domains demonstrate that our symmetry-aware training effectively and efficiently addresses the limitations of PlanGPT.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 19 of its 41 references that are in the map. First posted on arXiv 11 Aug 2025.

Builds on

19 of 41 references.

References · 19

  1. 2018How Powerful are Graph Neural Networks?cited by 4,393
  2. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  3. 2019Attention is not Explanationcited by 929
  4. 2023Large Language Modelscited by 780
  5. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 501
  6. 2023The Impact of Positional Encoding on Length Generalization in Transformerscited by 327
  7. 2014The LAMA Planner: Guiding Cost-Based Anytime Planning with Landmarkscited by 87
  8. 2024Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Tracescited by 43
  9. 2019ASNets: Deep Learning for Generalised Planningcited by 42
  10. 2022Plansformer: Generating Symbolic Plans using Transformerscited by 41
  11. 2021Learning General Optimal Policies with Graph Neural Networks: Expressive Power, Transparency, and Limitscited by 38
  12. 2024Thought of Search: Planning with Language Models Through The Lens of Efficiencycited by 32
7 more
  1. 2025Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challengescited by 31
  2. 2022Learning Generalized Policies Without Supervision Using GNNscited by 22
  3. 2024NL2Plan: Robust LLM-Driven Planning from Minimal Text Descriptionscited by 22
  4. 2024Return to Tradition: Learning Reliable Heuristics with Classical Machine Learningcited by 19
  5. 2024Planning in the Dark: LLM-Symbolic Planning Pipeline without Expertscited by 16
  6. 2025LLMs Can Plan Only If We Tell Themcited by 15
  7. 2024Learning More Expressive General Policies for Classical Planning Domainscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026On the Ability of Transformers to Verify Planscited by 2
  2. 2026Self-Improvement for Fast, High-Quality Plan Generationcited by 1
  3. 2026LLM-Evolved Domain-Independent Heuristics for Symbolic AI Planningcited by 1
  4. 2026On Sample-Efficient Generalized Planning via Learned Transition Modelscited by 0
  5. 2026Efficient Test-time Inference for Generative Planning Models with OCL Searchcited by 0