# Unsupervised Grounding of Plannable First-Order Logic Representation from Images

arXiv: https://arxiv.org/abs/1902.08093 · HTML: https://mlsanity.com/paper/1902.08093/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Feb 2019.

## Built on by (most cited first, 24 of 24)

- [Learning Neuro-Symbolic Relational Transition Models for Bilevel Planning](https://mlsanity.com/paper/2105.14074/) (2021)
- [Modular Design Patterns for Hybrid Learning and Reasoning Systems: a taxonomy, patterns and use cases](https://mlsanity.com/paper/2102.11965/) (2021)
- [Learning General Optimal Policies with Graph Neural Networks: Expressive Power, Transparency, and Limits](https://mlsanity.com/paper/2109.10129/) (2021)
- [From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence](https://mlsanity.com/paper/2110.15245/) (2021)
- [Learning First-Order Representations for Planning from Black-Box States: New Results](https://mlsanity.com/paper/2105.10830/) (2021)
- [Classical Planning in Deep Latent Space](https://mlsanity.com/paper/2107.00110/) (2021)
- [Learning Generalized Policies Without Supervision Using GNNs](https://mlsanity.com/paper/2205.06002/) (2022)
- [Expressing and Exploiting the Common Subgoal Structure of Classical Planning Domains Using Sketches: Extended Version](https://mlsanity.com/paper/2105.04250/) (2021)
- [pix2rule: End-to-end Neuro-symbolic Rule Learning](https://mlsanity.com/paper/2106.07487/) (2021)
- [Online Learning of Reusable Abstract Models for Object Goal Navigation](https://mlsanity.com/paper/2203.02583/) (2022)
- [Disentangled Relational Representations for Explaining and Learning from Demonstration](https://mlsanity.com/paper/1907.13627/) (2019)
- [Learning Lifted STRIPS Models from Action Traces Alone: A Simple, General, and Scalable Solution](https://mlsanity.com/paper/2411.14995/) (2024)
- [Natural Language-conditioned Reinforcement Learning with Inside-out Task Language Development and Translation](https://mlsanity.com/paper/2302.09368/) (2023)
- [Target Languages (vs. Inductive Biases) for Learning to Act and Plan](https://mlsanity.com/paper/2109.07195/) (2021)
- [Eye of the Beholder: Improved Relation Generalization for Text-based Reinforcement Learning Agents](https://mlsanity.com/paper/2106.05387/) (2021)
- [Learning Abstract World Model for Value-preserving Planning with Options](https://mlsanity.com/paper/2406.15850/) (2024)
- [Neuro-Symbolic Imitation Learning: Discovering Symbolic Abstractions for Skill Learning](https://mlsanity.com/paper/2503.21406/) (2025)
- [Help Me Explore: Minimal Social Interventions for Graph-Based Autotelic Agents](https://mlsanity.com/paper/2202.05129/) (2022)
- [Learning General Policies with Policy Gradient Methods](https://mlsanity.com/paper/2512.19366/) (2025)
- [Learning First-Order Symbolic Planning Representations That Are Grounded](https://mlsanity.com/paper/2204.11902/) (2022)
- [PlatoLTL: Scaling LTL-Guided Multi-Task RL](https://mlsanity.com/paper/2601.22891/) (2026)
- [Learning Visual Planning Models from Partially Observed Images](https://mlsanity.com/paper/2211.15666/) (2022)
- [Planning for Learning Object Properties](https://mlsanity.com/paper/2301.06054/) (2023)
- [Synthesizing Evolving Symbolic Representations for Autonomous Systems](https://mlsanity.com/paper/2409.11756/) (2024)

## Abstract (arXiv)

Recently, there is an increasing interest in obtaining the relational structures of the environment in the Reinforcement Learning community. However, the resulting "relations" are not the discrete, logical predicates compatible to the symbolic reasoning such as classical planning or goal recognition. Meanwhile, Latplan (Asai and Fukunaga 2018) bridged the gap between deep-learning perceptual systems and symbolic classical planners. One key component of the system is a Neural Network called State AutoEncoder (SAE), which encodes an image-based input into a propositional representation compatible to classical planning. To get the best of both worlds, we propose First-Order State AutoEncoder, an unsupervised architecture for grounding the first-order logic predicates and facts. Each predicate models a relationship between objects by taking the interpretable arguments and returning a propositional value. In the experiment using 8-Puzzle and a photo-realistic Blocksworld environment, we show that (1) the resulting predicates capture the interpretable relations (e.g. spatial), (2) they help obtaining the compact, abstract model of the environment, and finally, (3) the resulting model is compatible to symbolic classical planning.