# Compositional Scene Modeling with Global Object-Centric Representations

arXiv: https://arxiv.org/abs/2211.11500 · HTML: https://mlsanity.com/paper/2211.11500/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 51 references that are in the map.
- First posted on arXiv 21 Nov 2022.

## Builds on (most cited first, 30 of 30)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Multi-Object Representation Learning with Iterative Variational Inference](https://mlsanity.com/paper/1903.00450/) (2019)
- [GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations](https://mlsanity.com/paper/1907.13052/) (2019)
- [Attend, Infer, Repeat: Fast Scene Understanding with Generative Models](https://mlsanity.com/paper/1603.08575/) (2016)
- [SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition](https://mlsanity.com/paper/2001.02407/) (2020)
- [Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects](https://mlsanity.com/paper/1806.01794/) (2018)
- [Neural Expectation Maximization](https://mlsanity.com/paper/1708.03498/) (2017)
- [GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement](https://mlsanity.com/paper/2104.09958/) (2021)
- [Improving Generative Imagination in Object-Centric World Models](https://mlsanity.com/paper/2010.02054/) (2020)
- [Tagger: Deep Unsupervised Perceptual Grouping](https://mlsanity.com/paper/1606.06724/) (2016)
- [ShapeStacks: Learning Vision-Based Physical Intuition for Generalised Object Stacking](https://mlsanity.com/paper/1804.08018/) (2018)
- [Unsupervised Layered Image Decomposition into Object Prototypes](https://mlsanity.com/paper/2104.14575/) (2021)
- [Efficient Iterative Amortized Inference for Learning Symmetric and Disentangled Multi-Object Representations](https://mlsanity.com/paper/2106.03630/) (2021)
- [MarioNette: Self-Supervised Sprite Learning](https://mlsanity.com/paper/2104.14553/) (2021)
- [RELATE: Physically Plausible Multi-Object Scene Synthesis Using Structured Latent Spaces](https://mlsanity.com/paper/2007.01272/) (2020)
- [Compositional Scene Representation Learning via Reconstruction: A Survey](https://mlsanity.com/paper/2202.07135/) (2022)
- [Semi-convolutional Operators for Instance Segmentation](https://mlsanity.com/paper/1807.10712/) (2018)
- [Learning to Manipulate Individual Objects in an Image](https://mlsanity.com/paper/2004.05495/) (2020)
- [Binding via Reconstruction Clustering](https://mlsanity.com/paper/1511.06418/) (2015)
- [Unsupervised Learning of Compositional Scene Representations from Multiple Unspecified Viewpoints](https://mlsanity.com/paper/2112.03568/) (2021)
- [Spatial Mixture Models with Learnable Deep Priors for Perceptual Grouping](https://mlsanity.com/paper/1902.02502/) (2019)
- [Knowledge-Guided Object Discovery with Acquired Deep Impressions](https://mlsanity.com/paper/2103.10611/) (2021)
- [Unsupervised Image Decomposition with Phase-Correlation Networks](https://mlsanity.com/paper/2110.03473/) (2021)

## Built on by (most cited first, 1 of 1)

- [Learning Global Object-Centric Representations via Disentangled Slot Attention](https://mlsanity.com/paper/2410.18809/) (2024)

## Abstract (arXiv)

The appearance of the same object may vary in different scene images due to perspectives and occlusions between objects. Humans can easily identify the same object, even if occlusions exist, by completing the occluded parts based on its canonical image in the memory. Achieving this ability is still a challenge for machine learning, especially under the unsupervised learning setting. Inspired by such an ability of humans, this paper proposes a compositional scene modeling method to infer global representations of canonical images of objects without any supervision. The representation of each object is divided into an intrinsic part, which characterizes globally invariant information (i.e. canonical representation of an object), and an extrinsic part, which characterizes scene-dependent information (e.g., position and size). To infer the intrinsic representation of each object, we employ a patch-matching strategy to align the representation of a potentially occluded object with the canonical representations of objects, and sample the most probable canonical representation based on the category of object determined by amortized variational inference. Extensive experiments are conducted on four object-centric learning benchmarks, and experimental results demonstrate that the proposed method not only outperforms state-of-the-arts in terms of segmentation and reconstruction, but also achieves good global object identification performance.