# Structured agents for physical construction

arXiv: https://arxiv.org/abs/1904.03177 · HTML: https://mlsanity.com/paper/1904.03177/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Apr 2019.

## Built on by (most cited first, 30 of 50)

- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videos](https://mlsanity.com/paper/2205.14065/) (2022)
- [A Generalist Neural Algorithmic Learner](https://mlsanity.com/paper/2209.11142/) (2022)
- [On Evaluation Metrics for Graph Generative Models](https://mlsanity.com/paper/2201.09871/) (2022)
- [Learning Object-Centric Representations of Multi-Object Scenes from Multiple Views](https://mlsanity.com/paper/2111.07117/) (2021)
- [Procedural Generalization by Planning with Self-Supervised World Models](https://mlsanity.com/paper/2111.01587/) (2021)
- [Is an Object-Centric Video Representation Beneficial for Transfer?](https://mlsanity.com/paper/2207.10075/) (2022)
- [Policy Architectures for Compositional Generalization in Control](https://mlsanity.com/paper/2203.05960/) (2022)
- [Blocks Assemble! Learning to Assemble with Large-Scale Structured Reinforcement Learning](https://mlsanity.com/paper/2203.13733/) (2022)
- [IKEA-Manual: Seeing Shape Assembly Step by Step](https://mlsanity.com/paper/2302.01881/) (2023)
- [Systematic Visual Reasoning through Object-Centric Relational Abstraction](https://mlsanity.com/paper/2306.02500/) (2023)
- [Brick-by-Brick: Combinatorial Construction with Deep Reinforcement Learning](https://mlsanity.com/paper/2110.15481/) (2021)
- [Causal-based Time Series Domain Generalization for Vehicle Intention Prediction](https://mlsanity.com/paper/2112.02093/) (2021)
- [Structure in Deep Reinforcement Learning: A Survey and Open Problems](https://mlsanity.com/paper/2306.16021/) (2023)
- [Graph Reinforcement Learning for Combinatorial Optimization: A Survey and Unifying Perspective](https://mlsanity.com/paper/2404.06492/) (2024)
- [Learning to communicate about shared procedural abstractions](https://mlsanity.com/paper/2107.00077/) (2021)
- [Bi-Manual Manipulation and Attachment via Sim-to-Real Reinforcement Learning](https://mlsanity.com/paper/2203.08277/) (2022)
- [Binding Actions to Objects in World Models](https://mlsanity.com/paper/2204.13022/) (2022)
- [Break and Make: Interactive Structural Understanding Using LEGO Bricks](https://mlsanity.com/paper/2207.13738/) (2022)
- [Recurrent Complex-Weighted Autoencoders for Unsupervised Object Discovery](https://mlsanity.com/paper/2405.17283/) (2024)
- [Snowflake: Scaling GNNs to High-Dimensional Continuous Control via Parameter Freezing](https://mlsanity.com/paper/2103.01009/) (2021)
- [Survey of Image Based Graph Neural Networks](https://mlsanity.com/paper/2106.06307/) (2021)
- [Graph-based Reinforcement Learning meets Mixed Integer Programs: An application to 3D robot assembly discovery](https://mlsanity.com/paper/2203.04120/) (2022)
- [Geoclidean: Few-Shot Generalization in Euclidean Geometry](https://mlsanity.com/paper/2211.16663/) (2022)
- [Neural Constraint Satisfaction: Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement](https://mlsanity.com/paper/2303.11373/) (2023)
- [Physics-Aware Combinatorial Assembly Sequence Planning using Data-free Action Masking](https://mlsanity.com/paper/2408.10162/) (2024)
- [Creative Problem Solving in Artificially Intelligent Agents: A Survey and Framework](https://mlsanity.com/paper/2204.10358/) (2022)
- [Bi-Manual Block Assembly via Sim-to-Real Reinforcement Learning](https://mlsanity.com/paper/2303.14870/) (2023)

## Abstract (arXiv)

Physical construction---the ability to compose objects, subject to physical dynamics, to serve some function---is fundamental to human intelligence. We introduce a suite of challenging physical construction tasks inspired by how children play with blocks, such as matching a target configuration, stacking blocks to connect objects together, and creating shelter-like structures over target objects. We examine how a range of deep reinforcement learning agents fare on these challenges, and introduce several new approaches which provide superior performance. Our results show that agents which use structured representations (e.g., objects and scene graphs) and structured policies (e.g., object-centric actions) outperform those which use less structured representations, and generalize better beyond their training when asked to reason about larger scenes. Model-based agents which use Monte-Carlo Tree Search also outperform strictly model-free agents in our most challenging construction problems. We conclude that approaches which combine structured representations and reasoning with powerful learning are a key path toward agents that possess rich intuitive physics, scene understanding, and planning.