# Unsupervised Learning of Compositional Scene Representations from Multiple Unspecified Viewpoints

arXiv: https://arxiv.org/abs/2112.03568 · HTML: https://mlsanity.com/paper/2112.03568/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 7 Dec 2021.

## Builds on (most cited first, 28 of 28)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Multi-Object Representation Learning with Iterative Variational Inference](https://mlsanity.com/paper/1903.00450/) (2019)
- [GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations](https://mlsanity.com/paper/1907.13052/) (2019)
- [Attend, Infer, Repeat: Fast Scene Understanding with Generative Models](https://mlsanity.com/paper/1603.08575/) (2016)
- [SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition](https://mlsanity.com/paper/2001.02407/) (2020)
- [Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects](https://mlsanity.com/paper/1806.01794/) (2018)
- [Neural Expectation Maximization](https://mlsanity.com/paper/1708.03498/) (2017)
- [Neural Variational Inference and Learning in Belief Networks](https://mlsanity.com/paper/1402.0030/) (2014)
- [Entity Abstraction in Visual Model-Based Reinforcement Learning](https://mlsanity.com/paper/1910.12827/) (2019)
- [SCALOR: Generative World Models with Scalable Object Representations](https://mlsanity.com/paper/1910.02384/) (2019)
- [Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions](https://mlsanity.com/paper/1802.10353/) (2018)
- [Iterative Amortized Inference](https://mlsanity.com/paper/1807.09356/) (2018)
- [Generative Neurosymbolic Machines](https://mlsanity.com/paper/2010.12152/) (2020)
- [Exploiting Spatial Invariance for Scalable Unsupervised Object Tracking](https://mlsanity.com/paper/1911.09033/) (2019)
- [ROOTS: Object-Centric Representation and Rendering of 3D Scenes](https://mlsanity.com/paper/2006.06130/) (2020)
- [Learning Object-Centric Representations of Multi-Object Scenes from Multiple Views](https://mlsanity.com/paper/2111.07117/) (2021)
- [Efficient Iterative Amortized Inference for Learning Symmetric and Disentangled Multi-Object Representations](https://mlsanity.com/paper/2106.03630/) (2021)
- [Benchmarking Unsupervised Object Representations for Video Sequences](https://mlsanity.com/paper/2006.07034/) (2020)
- [Tracking by Animation: Unsupervised Learning of Multi-Object Attentive Trackers](https://mlsanity.com/paper/1809.03137/) (2018)
- [R-SQAIR: Relational Sequential Attend, Infer, Repeat](https://mlsanity.com/paper/1910.05231/) (2019)
- [Efficient inference in occlusion-aware generative models of images](https://mlsanity.com/paper/1511.06362/) (2015)
- [Spatial Mixture Models with Learnable Deep Priors for Perceptual Grouping](https://mlsanity.com/paper/1902.02502/) (2019)
- [Knowledge-Guided Object Discovery with Acquired Deep Impressions](https://mlsanity.com/paper/2103.10611/) (2021)

## Built on by (most cited first, 11 of 11)

- [Compositional Scene Representation Learning via Reconstruction: A Survey](https://mlsanity.com/paper/2202.07135/) (2022)
- [On permutation-invariant neural networks](https://mlsanity.com/paper/2403.17410/) (2024)
- [Unsupervised Object-Centric Learning from Multiple Unspecified Viewpoints](https://mlsanity.com/paper/2401.01922/) (2024)
- [Compositional Law Parsing with Latent Random Functions](https://mlsanity.com/paper/2209.09115/) (2022)
- [Time-Conditioned Generative Modeling of Object-Centric Representations for Video Decomposition and Prediction](https://mlsanity.com/paper/2301.08951/) (2023)
- [OCTScenes: A Versatile Real-World Dataset of Tabletop Scenes for Object-Centric Learning](https://mlsanity.com/paper/2306.09682/) (2023)
- [Towards Generative Abstract Reasoning: Completing Raven's Progressive Matrix via Rule Abstraction and Selection](https://mlsanity.com/paper/2401.09966/) (2024)
- [Towards Improving the Generation Quality of Autoregressive Slot VAEs](https://mlsanity.com/paper/2206.01370/) (2022)
- [Compositional Scene Modeling with Global Object-Centric Representations](https://mlsanity.com/paper/2211.11500/) (2022)
- [Improving Viewpoint-Independent Object-Centric Representations through Active Viewpoint Selection](https://mlsanity.com/paper/2411.00402/) (2024)
- [ParticleSplat: Self-supervised Object-centric Latent Particle Splatting](https://mlsanity.com/paper/2609.19463/) (2026)

## Abstract (arXiv)

Visual scenes are extremely rich in diversity, not only because there are infinite combinations of objects and background, but also because the observations of the same scene may vary greatly with the change of viewpoints. When observing a visual scene that contains multiple objects from multiple viewpoints, humans are able to perceive the scene in a compositional way from each viewpoint, while achieving the so-called "object constancy" across different viewpoints, even though the exact viewpoints are untold. This ability is essential for humans to identify the same object while moving and to learn from vision efficiently. It is intriguing to design models that have the similar ability. In this paper, we consider a novel problem of learning compositional scene representations from multiple unspecified viewpoints without using any supervision, and propose a deep generative model which separates latent representations into a viewpoint-independent part and a viewpoint-dependent part to solve this problem. To infer latent representations, the information contained in different viewpoints is iteratively integrated by neural networks. Experiments on several specifically designed synthetic datasets have shown that the proposed method is able to effectively learn from multiple unspecified viewpoints.