# Constellation: Learning relational abstractions over objects for compositional imagination

arXiv: https://arxiv.org/abs/2107.11153 · HTML: https://mlsanity.com/paper/2107.11153/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 21 references that are in the map.
- First posted on arXiv 23 Jul 2021.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Are Disentangled Representations Helpful for Abstract Visual Reasoning?](https://mlsanity.com/paper/1905.12506/) (2019)
- [Taming VAEs](https://mlsanity.com/paper/1810.00597/) (2018)
- [Stacked Capsule Autoencoders](https://mlsanity.com/paper/1906.06818/) (2019)
- [On the Transfer of Disentangled Representations in Realistic Settings](https://mlsanity.com/paper/2010.14407/) (2020)

## Built on by (most cited first, 3 of 3)

- [Disentanglement with Biological Constraints: A Theory of Functional Cell Types](https://mlsanity.com/paper/2210.01768/) (2022)
- [When does compositional structure yield compositional generalization? A kernel theory](https://mlsanity.com/paper/2405.16391/) (2024)
- [Im-Promptu: In-Context Composition from Image Prompts](https://mlsanity.com/paper/2305.17262/) (2023)

## Abstract (arXiv)

Learning structured representations of visual scenes is currently a major bottleneck to bridging perception with reasoning. While there has been exciting progress with slot-based models, which learn to segment scenes into sets of objects, learning configurational properties of entire groups of objects is still under-explored. To address this problem, we introduce Constellation, a network that learns relational abstractions of static visual scenes, and generalises these abstractions over sensory particularities, thus offering a potential basis for abstract relational reasoning. We further show that this basis, along with language association, provides a means to imagine sensory content in new ways. This work is a first step in the explicit representation of visual relationships and using them for complex cognitive procedures.