mlsanityData to 14 Oct 2026

Learning Planning Abstractions from Language

arXiv 2405.03864 ↗First posted 6 May 2024cs.RO cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

This paper presents a framework for learning state and action abstractions in sequential decision-making domains. Our framework, planning abstraction from language (PARL), utilizes language-annotated demonstrations to automatically discover a symbolic and abstract action space and induce a latent state abstraction based on it. PARL consists of three stages: 1) recovering object-level and action concepts, 2) learning state abstractions, abstract action feasibility, and transition models, and 3) applying low-level policies for abstract actions. During inference, given the task description, PARL first makes abstract action plans using the latent transition and feasibility functions, then refines the high-level plan using low-level policies. PARL generalizes across scenarios involving novel object instances and environments, unseen concept compositions, and tasks that require longer planning horizons than settings it is trained on.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 37 of its 62 references that are in the map. First posted on arXiv 6 May 2024.

Builds on

37 of 62 references.

References · 37

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  6. 2017Hindsight Experience Replaycited by 1,136
  7. 2020PCT: Point cloud transformercited by 732
  8. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  9. 2022GLIPv2: Unifying Localization and Vision-Language Understandingcited by 275
  10. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 203
  11. 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 188
  12. 2023Language-Driven Representation Learning for Roboticscited by 165
25 more
  1. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  2. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  3. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Datacited by 138
  4. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 118
  5. 2021Skill Induction and Planning with Latent Languagecited by 99
  6. 2022Object Scene Representation Transformercited by 90
  7. 2017Mapping Instructions and Visual Observations to Actions with Reinforcement Learningcited by 81
  8. 2017Recurrent Environment Simulatorscited by 68
  9. 2021StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objectscited by 59
  10. 2021Learning Symbolic Operators for Task and Motion Planningcited by 58
  11. 2015Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequencescited by 50
  12. 2021PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D Worldcited by 47
  13. 2021Learning Neuro-Symbolic Relational Transition Models for Bilevel Planningcited by 43
  14. 2019Learning First-Order Symbolic Representations for Planning from the Structure of the State Spacecited by 38
  15. 2019Regression Planning Networkscited by 32
  16. 2021Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoningcited by 26
  17. 2023Programmatically Grounded, Compositionally Generalizable Robotic Manipulationcited by 26
  18. 2015Alignment-based compositional semantics for instruction followingcited by 21
  19. 2022Generalizable Task Planning through Representation Pretrainingcited by 19
  20. 2022Sequence-Based Plan Feasibility Prediction for Efficient Task and Motion Planningcited by 19
  21. 2020Language-Mediated, Object-Centric Representation Learningcited by 11
  22. 2020Learning Neural-Symbolic Descriptive Planning Models via Cube-Space Priors: The Voyage Home (to STRIPS)cited by 7
  23. 2021Learning and Planning for Temporally Extended Tasks in Unknown Environmentscited by 7
  24. 2023Latent Space Planning for Multi-Object Manipulation with Environment-Aware Relational Classifierscited by 6
  25. 2023Learning Rational Subgoals from Demonstrations and Instructionscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024LogiCity: Advancing Neuro-Symbolic AI with Abstract Urban Simulationcited by 8
  2. 2025MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulationcited by 7
  3. 2026DreamProver: Evolving Transferable Lemma Libraries via a Wake-Sleep Theorem-Proving Agentcited by 5