# Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundary

arXiv: https://arxiv.org/abs/1705.00154 · HTML: https://mlsanity.com/paper/1705.00154/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2017.

## Built on by (most cited first, 30 of 58)

- [A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning](https://mlsanity.com/paper/2110.01515/) (2021)
- [Neuro-Symbolic Artificial Intelligence: Current Trends](https://mlsanity.com/paper/2105.05330/) (2021)
- [Predicate Invention for Bilevel Planning](https://mlsanity.com/paper/2203.09634/) (2022)
- [Learning Neuro-Symbolic Relational Transition Models for Bilevel Planning](https://mlsanity.com/paper/2105.14074/) (2021)
- [World Models and Predictive Coding for Cognitive and Developmental Robotics: Frontiers and Challenges](https://mlsanity.com/paper/2301.05832/) (2023)
- [From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence](https://mlsanity.com/paper/2110.15245/) (2021)
- [Model-Based Reinforcement Learning via Latent-Space Collocation](https://mlsanity.com/paper/2106.13229/) (2021)
- [Classical Planning in Deep Latent Space](https://mlsanity.com/paper/2107.00110/) (2021)
- [Subgoal Search For Complex Reasoning Tasks](https://mlsanity.com/paper/2108.11204/) (2021)
- [Learning Efficient Abstract Planning Models that Choose What to Predict](https://mlsanity.com/paper/2208.07737/) (2022)
- [Neural Concept Binder](https://mlsanity.com/paper/2406.09949/) (2024)
- [Inductive Biases for Object-Centric Representations in the Presence of Complex Textures](https://mlsanity.com/paper/2204.08479/) (2022)
- [Neuro-algorithmic Policies enable Fast Combinatorial Generalization](https://mlsanity.com/paper/2102.07456/) (2021)
- [From Pixels to Predicates: Learning Symbolic World Models via Pretrained Vision-Language Models](https://mlsanity.com/paper/2501.00296/) (2024)
- [pix2rule: End-to-end Neuro-symbolic Rule Learning](https://mlsanity.com/paper/2106.07487/) (2021)
- [Online Learning of Reusable Abstract Models for Object Goal Navigation](https://mlsanity.com/paper/2203.02583/) (2022)
- [Disentangled Relational Representations for Explaining and Learning from Demonstration](https://mlsanity.com/paper/1907.13627/) (2019)
- [Learning Lifted STRIPS Models from Action Traces Alone: A Simple, General, and Scalable Solution](https://mlsanity.com/paper/2411.14995/) (2024)
- [Heuristic Search Planning with Deep Neural Networks using Imitation, Attention and Curriculum Learning](https://mlsanity.com/paper/2112.01918/) (2021)
- [MinePlanner: A Benchmark for Long-Horizon Planning in Large Minecraft Worlds](https://mlsanity.com/paper/2312.12891/) (2023)
- [Planning from Pixels in Environments with Combinatorially Hard Search Spaces](https://mlsanity.com/paper/2110.06149/) (2021)
- [ODGR: Online Dynamic Goal Recognition](https://mlsanity.com/paper/2407.16220/) (2024)
- [Eye of the Beholder: Improved Relation Generalization for Text-based Reinforcement Learning Agents](https://mlsanity.com/paper/2106.05387/) (2021)
- [Learning Abstract World Model for Value-preserving Planning with Options](https://mlsanity.com/paper/2406.15850/) (2024)
- [Neuro-Symbolic Imitation Learning: Discovering Symbolic Abstractions for Skill Learning](https://mlsanity.com/paper/2503.21406/) (2025)
- [Learning Temporally Extended Skills in Continuous Domains as Symbolic Actions for Planning](https://mlsanity.com/paper/2207.05018/) (2022)
- [Discovering Predictive Relational Object Symbols with Symbolic Attentive Layers](https://mlsanity.com/paper/2309.00889/) (2023)
- [SLAP: Shortcut Learning for Abstract Planning](https://mlsanity.com/paper/2511.01107/) (2025)
- [Learning Lifted Action Models from Unsupervised Visual Traces](https://mlsanity.com/paper/2604.19043/) (2026)
- [High-level Features for Resource Economy and Fast Learning in Skill Transfer](https://mlsanity.com/paper/2106.10354/) (2021)

## Abstract (arXiv)

Current domain-independent, classical planners require symbolic models of the problem domain and instance as input, resulting in a knowledge acquisition bottleneck. Meanwhile, although deep learning has achieved significant success in many fields, the knowledge is encoded in a subsymbolic representation which is incompatible with symbolic systems such as planners. We propose LatPlan, an unsupervised architecture combining deep learning and classical planning. Given only an unlabeled set of image pairs showing a subset of transitions allowed in the environment (training inputs), and a pair of images representing the initial and the goal states (planning inputs), LatPlan finds a plan to the goal state in a symbolic latent space and returns a visualized plan execution. The contribution of this paper is twofold: (1) State Autoencoder, which finds a propositional state representation of the environment using a Variational Autoencoder. It generates a discrete latent vector from the images, based on which a PDDL model can be constructed and then solved by an off-the-shelf planner. (2) Action Autoencoder / Discriminator, a neural architecture which jointly finds the action symbols and the implicit action models (preconditions/effects), and provides a successor function for the implicit graph search. We evaluate LatPlan using image-based versions of 3 planning domains: 8-puzzle, Towers of Hanoi and LightsOut.