# Discovering objects and their relations from entangled scene representations

arXiv: https://arxiv.org/abs/1702.05068 · HTML: https://mlsanity.com/paper/1702.05068/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2017.

## Built on by (most cited first, 12 of 12)

- [Neural Production Systems: Learning Rule-Governed Visual Dynamics](https://mlsanity.com/paper/2103.01937/) (2021)
- [Learning to Compose Visual Relations](https://mlsanity.com/paper/2111.09297/) (2021)
- [Discrete-Valued Neural Communication](https://mlsanity.com/paper/2107.02367/) (2021)
- [Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understanding](https://mlsanity.com/paper/2312.05328/) (2023)
- [The Power of the Weisfeiler-Leman Algorithm for Machine Learning with Graphs](https://mlsanity.com/paper/2105.05911/) (2021)
- [Compositional Multi-Object Reinforcement Learning with Linear Relation Networks](https://mlsanity.com/paper/2201.13388/) (2022)
- [A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2206.04551/) (2022)
- [Topos and Stacks of Deep Neural Networks](https://mlsanity.com/paper/2106.14587/) (2021)
- [Scaling Graph-based Deep Learning models to larger networks](https://mlsanity.com/paper/2110.01261/) (2021)
- [From Images to Connections: Can DQN with GNNs learn the Strategic Game of Hex?](https://mlsanity.com/paper/2311.13414/) (2023)
- [β-Multivariational Autoencoder for Entangled Representation Learning in Video Frames](https://mlsanity.com/paper/2211.12627/) (2022)
- [Relational Structural Causal Models](https://mlsanity.com/paper/2606.14892/) (2026)

## Abstract (arXiv)

Our world can be succinctly and compactly described as structured scenes of objects and relations. A typical room, for example, contains salient objects such as tables, chairs and books, and these objects typically relate to each other by their underlying causes and semantics. This gives rise to correlated features, such as position, function and shape. Humans exploit knowledge of objects and their relations for learning a wide spectrum of tasks, and more generally when learning the structure underlying observed data. In this work, we introduce relation networks (RNs) - a general purpose neural network architecture for object-relation reasoning. We show that RNs are capable of learning object relations from scene description data. Furthermore, we show that RNs can act as a bottleneck that induces the factorization of objects from entangled scene description inputs, and from distributed deep representations of scene images provided by a variational autoencoder. The model can also be used in conjunction with differentiable memory mechanisms for implicit relation discovery in one-shot learning tasks. Our results suggest that relation networks are a potentially powerful architecture for solving a variety of problems that require object relation reasoning.