# Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions

arXiv: https://arxiv.org/abs/1802.10353 · HTML: https://mlsanity.com/paper/1802.10353/

## Key facts

- Cited by 100 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2018.

## Built on by (most cited first, 30 of 100)

- [Conditional Object-Centric Learning from Video](https://mlsanity.com/paper/2111.12594/) (2021)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2103.09504/) (2021)
- [SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videos](https://mlsanity.com/paper/2206.07764/) (2022)
- [SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models](https://mlsanity.com/paper/2210.05861/) (2022)
- [Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videos](https://mlsanity.com/paper/2205.14065/) (2022)
- [GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement](https://mlsanity.com/paper/2104.09958/) (2021)
- [Decomposing 3D Scenes into Objects via Unsupervised Volume Segmentation](https://mlsanity.com/paper/2104.01148/) (2021)
- [Coordination Among Neural Modules Through a Shared Global Workspace](https://mlsanity.com/paper/2103.01197/) (2021)
- [SlotDiffusion: Object-Centric Generative Modeling with Diffusion Models](https://mlsanity.com/paper/2305.11281/) (2023)
- [Generalization and Robustness Implications in Object-Centric Learning](https://mlsanity.com/paper/2107.00637/) (2021)
- [Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similarities](https://mlsanity.com/paper/2306.04829/) (2023)
- [Is Attention Better Than Matrix Decomposition?](https://mlsanity.com/paper/2109.04553/) (2021)
- [Neural Production Systems: Learning Rule-Governed Visual Dynamics](https://mlsanity.com/paper/2103.01937/) (2021)
- [Unsupervised Learning of Compositional Energy Concepts](https://mlsanity.com/paper/2111.03042/) (2021)
- [Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning](https://mlsanity.com/paper/2107.00848/) (2021)
- [Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Models](https://mlsanity.com/paper/2205.13817/) (2022)
- [Invariant Slot Attention: Object Discovery with Slot-Centric Reference Frames](https://mlsanity.com/paper/2302.04973/) (2023)
- [Unsupervised Layered Image Decomposition into Object Prototypes](https://mlsanity.com/paper/2104.14575/) (2021)
- [Object Representations as Fixed Points: Training Iterative Refinement Algorithms with Implicit Differentiation](https://mlsanity.com/paper/2207.00787/) (2022)
- [Provably Learning Object-Centric Representations](https://mlsanity.com/paper/2305.14229/) (2023)
- [iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesis](https://mlsanity.com/paper/2107.02790/) (2021)
- [Discrete-Valued Neural Communication](https://mlsanity.com/paper/2107.02367/) (2021)
- [Learning to Accelerate Partial Differential Equations via Latent Global Evolution](https://mlsanity.com/paper/2206.07681/) (2022)
- [Understanding Object Dynamics for Interactive Image-to-Video Synthesis](https://mlsanity.com/paper/2106.11303/) (2021)
- [Neural Systematic Binder](https://mlsanity.com/paper/2211.01177/) (2022)
- [Efficient Iterative Amortized Inference for Learning Symmetric and Disentangled Multi-Object Representations](https://mlsanity.com/paper/2106.03630/) (2021)
- [RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting](https://mlsanity.com/paper/2108.01316/) (2021)
- [PartAfford: Part-level Affordance Discovery from 3D Objects](https://mlsanity.com/paper/2202.13519/) (2022)
- [VAEL: Bridging Variational Autoencoders and Probabilistic Logic Programming](https://mlsanity.com/paper/2202.04178/) (2022)

## Abstract (arXiv)

Common-sense physical reasoning is an essential ingredient for any intelligent agent operating in the real-world. For example, it can be used to simulate the environment, or to infer the state of parts of the world that are currently unobserved. In order to match real-world conditions this causal knowledge must be learned without access to supervised data. To address this problem we present a novel method that learns to discover objects and model their physical interactions from raw visual images in a purely unsupervised fashion. It incorporates prior knowledge about the compositional nature of human perception to factor interactions between object-pairs and learn efficiently. On videos of bouncing balls we show the superior modelling capabilities of our method compared to other unsupervised neural approaches that do not incorporate such prior knowledge. We demonstrate its ability to handle occlusion and show that it can extrapolate learned knowledge to scenes with different numbers of objects.