# Embodied Active Learning of Relational State Abstractions for Bilevel Planning

arXiv: https://arxiv.org/abs/2303.04912 · HTML: https://mlsanity.com/paper/2303.04912/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 69 references that are in the map.
- First posted on arXiv 8 Mar 2023.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Self-Supervised Exploration via Disagreement](https://mlsanity.com/paper/1906.04161/) (2019)
- [Pre-Trained Language Models for Interactive Decision-Making](https://mlsanity.com/paper/2202.01771/) (2022)
- [Skill Induction and Planning with Latent Language](https://mlsanity.com/paper/2110.01517/) (2021)
- [A Deep Hierarchical Approach to Lifelong Learning in Minecraft](https://mlsanity.com/paper/1604.07255/) (2016)
- [Batch Active Preference-Based Learning of Reward Functions](https://mlsanity.com/paper/1810.04303/) (2018)
- [On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)](https://mlsanity.com/paper/2302.06706/) (2023)
- [CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning](https://mlsanity.com/paper/1810.06284/) (2018)
- [Interactive Perception: Leveraging Action in Perception and Perception in Action](https://mlsanity.com/paper/1604.03670/) (2016)
- [Near Optimal Behavior via Approximate State Abstraction](https://mlsanity.com/paper/1701.04113/) (2017)
- [Learning Symbolic Operators for Task and Motion Planning](https://mlsanity.com/paper/2103.00589/) (2021)
- [Learning Neuro-Symbolic Skills for Bilevel Planning](https://mlsanity.com/paper/2206.10680/) (2022)
- [Predicate Invention for Bilevel Planning](https://mlsanity.com/paper/2203.09634/) (2022)
- [Learning compositional models of robot skills for task and motion planning](https://mlsanity.com/paper/2006.06444/) (2020)
- [SORNet: Spatial Object-Centric Representations for Sequential Manipulation](https://mlsanity.com/paper/2109.03891/) (2021)
- [Learning Neuro-Symbolic Relational Transition Models for Bilevel Planning](https://mlsanity.com/paper/2105.14074/) (2021)
- [Deep Affordance Foresight: Planning Through What Can Be Done in the Future](https://mlsanity.com/paper/2011.08424/) (2020)
- [Learning to Look Around: Intelligently Exploring Unseen Environments for Unknown Tasks](https://mlsanity.com/paper/1709.00507/) (2017)
- [Reset-Free Lifelong Learning with Skill-Space Planning](https://mlsanity.com/paper/2012.03548/) (2020)
- [Grounding Predicates through Actions](https://mlsanity.com/paper/2109.14718/) (2021)
- [Active Reinforcement Learning: Observing Rewards at a Cost](https://mlsanity.com/paper/2011.06709/) (2020)
- [Generalizable Task Planning through Representation Pretraining](https://mlsanity.com/paper/2205.07993/) (2022)
- [Active Reinforcement Learning with Monte-Carlo Tree Search](https://mlsanity.com/paper/1803.04926/) (2018)
- [Visual Curiosity: Learning to Ask Questions to Learn Visual Recognition](https://mlsanity.com/paper/1810.00912/) (2018)
- [Active Learning of Abstract Plan Feasibility](https://mlsanity.com/paper/2107.00683/) (2021)
- [Active Exploration for Learning Symbolic Representations](https://mlsanity.com/paper/1709.01490/) (2017)

## Built on by (most cited first, 8 of 8)

- [Practice Makes Perfect: Planning to Learn Skill Parameter Policies](https://mlsanity.com/paper/2402.15025/) (2024)
- [Synthesizing world models for bilevel planning](https://mlsanity.com/paper/2503.20124/) (2025)
- [Prime the search: Using large language models for guiding geometric task and motion planning by warm-starting tree search](https://mlsanity.com/paper/2506.07062/) (2025)
- [Discovering Predictive Relational Object Symbols with Symbolic Attentive Layers](https://mlsanity.com/paper/2309.00889/) (2023)
- [Few-Shot Neuro-Symbolic Imitation Learning for Long-Horizon Planning and Acting](https://mlsanity.com/paper/2508.21501/) (2025)
- [Coloring Between the Lines: Personalization in the Null Space of Planning Constraints](https://mlsanity.com/paper/2505.15503/) (2025)
- [ASkDAgger: Active Skill-level Data Aggregation for Interactive Imitation Learning](https://mlsanity.com/paper/2508.05310/) (2025)
- [Neuro-Symbolic Learning for Long-Horizon Task Planning Under Complex Logical Constraints](https://mlsanity.com/paper/2606.06877/) (2026)

## Abstract (arXiv)

State abstraction is an effective technique for planning in robotics environments with continuous states and actions, long task horizons, and sparse feedback. In object-oriented environments, predicates are a particularly useful form of state abstraction because of their compatibility with symbolic planners and their capacity for relational generalization. However, to plan with predicates, the agent must be able to interpret them in continuous environment states (i.e., ground the symbols). Manually programming predicate interpretations can be difficult, so we would instead like to learn them from data. We propose an embodied active learning paradigm where the agent learns predicate interpretations through online interaction with an expert. For example, after taking actions in a block stacking environment, the agent may ask the expert: "Is On(block1, block2) true?" From this experience, the agent learns to plan: it learns neural predicate interpretations, symbolic planning operators, and neural samplers that can be used for bilevel planning. During exploration, the agent plans to learn: it uses its current models to select actions towards generating informative expert queries. We learn predicate interpretations as ensembles of neural networks and use their entropy to measure the informativeness of potential queries. We evaluate this approach in three robotic environments and find that it consistently outperforms six baselines while exhibiting sample efficiency in two key metrics: number of environment interactions, and number of queries to the expert. Code: https://tinyurl.com/active-predicates