# Bridging the Gap to Real-World Object-Centric Learning

arXiv: https://arxiv.org/abs/2209.14860 · HTML: https://mlsanity.com/paper/2209.14860/

## Key facts

- Cited by 220 papers in the map (data through 14 Oct 2026).
- Builds on 73 of its 99 references that are in the map.
- First posted on arXiv 29 Sep 2022.

## Builds on (most cited first, 30 of 73)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [A Generalist Agent](https://mlsanity.com/paper/2205.06175/) (2022)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Perceiver IO: A General Architecture for Structured Inputs & Outputs](https://mlsanity.com/paper/2107.14795/) (2021)
- [GroupViT: Semantic Segmentation Emerges from Text Supervision](https://mlsanity.com/paper/2202.11094/) (2022)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Kubric: A scalable dataset generator](https://mlsanity.com/paper/2203.03570/) (2022)
- [Multi-Object Representation Learning with Iterative Variational Inference](https://mlsanity.com/paper/1903.00450/) (2019)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [Masked Siamese Networks for Label-Efficient Learning](https://mlsanity.com/paper/2204.07141/) (2022)
- [Conditional Object-Centric Learning from Video](https://mlsanity.com/paper/2111.12594/) (2021)
- [Unsupervised Semantic Segmentation by Distilling Feature Correspondences](https://mlsanity.com/paper/2203.08414/) (2022)
- [On the Binding Problem in Artificial Neural Networks](https://mlsanity.com/paper/2012.05208/) (2020)

## Built on by (most cited first, 30 of 220)

- [Disentangled Representation Learning](https://mlsanity.com/paper/2211.11695/) (2022)
- [Object-Centric Slot Diffusion](https://mlsanity.com/paper/2303.10834/) (2023)
- [SlotDiffusion: Object-Centric Generative Modeling with Diffusion Models](https://mlsanity.com/paper/2305.11281/) (2023)
- [Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similarities](https://mlsanity.com/paper/2306.04829/) (2023)
- [Multi-View Causal Representation Learning with Partial Observability](https://mlsanity.com/paper/2311.04056/) (2023)
- [Unsupervised Object Localization: Observing the Background to Discover Objects](https://mlsanity.com/paper/2212.07834/) (2022)
- [DiffusionSeg: Adapting Diffusion Towards Unsupervised Object Discovery](https://mlsanity.com/paper/2303.09813/) (2023)
- [Invariant Slot Attention: Object Discovery with Slot-Centric Reference Frames](https://mlsanity.com/paper/2302.04973/) (2023)
- [Self-supervised Object-Centric Learning for Videos](https://mlsanity.com/paper/2310.06907/) (2023)
- [SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformers](https://mlsanity.com/paper/2312.00648/) (2023)
- [Provably Learning Object-Centric Representations](https://mlsanity.com/paper/2305.14229/) (2023)
- [Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imagination](https://mlsanity.com/paper/2412.14957/) (2024)
- [EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Models](https://mlsanity.com/paper/2401.11739/) (2024)
- [Provable Compositional Generalization for Object-Centric Learning](https://mlsanity.com/paper/2310.05327/) (2023)
- [In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2408.04961/) (2024)
- [Artificial Kuramoto Oscillatory Neurons](https://mlsanity.com/paper/2410.13821/) (2024)
- [Time Does Tell: Self-Supervised Time-Tuning of Dense Image Representations](https://mlsanity.com/paper/2308.11796/) (2023)
- [Adaptive Slot Attention: Object Discovery with Dynamic Slot Number](https://mlsanity.com/paper/2406.09196/) (2024)
- [Temporally Consistent Object-Centric Learning by Contrasting Slots](https://mlsanity.com/paper/2412.14295/) (2024)
- [What is the Visual Cognition Gap between Humans and Multimodal LLMs?](https://mlsanity.com/paper/2406.10424/) (2024)
- [Rotating Features for Object Discovery](https://mlsanity.com/paper/2306.00600/) (2023)
- [EAGLE: Eigen Aggregation Learning for Object-Centric Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2403.01482/) (2024)
- [Object Discovery from Motion-Guided Tokens](https://mlsanity.com/paper/2303.15555/) (2023)
- [Shatter and Gather: Learning Referring Image Segmentation with Text Supervision](https://mlsanity.com/paper/2308.15512/) (2023)
- [Distilling Spectral Graph for Object-Context Aware Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2411.17150/) (2024)
- [Understanding Video Transformers via Universal Concept Discovery](https://mlsanity.com/paper/2401.10831/) (2024)
- [Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning](https://mlsanity.com/paper/2301.05169/) (2023)
- [Object-centric architectures enable efficient causal representation learning](https://mlsanity.com/paper/2310.19054/) (2023)
- [Systematic Visual Reasoning through Object-Centric Relational Abstraction](https://mlsanity.com/paper/2306.02500/) (2023)
- [Slot-VAE: Object-Centric Scene Generation with Slot Attention](https://mlsanity.com/paper/2306.06997/) (2023)

## Abstract (arXiv)

Humans naturally decompose their environment into entities at the appropriate level of abstraction to act in the world. Allowing machine learning algorithms to derive this decomposition in an unsupervised way has become an important line of research. However, current methods are restricted to simulated data or require additional information in the form of motion or depth in order to successfully discover objects. In this work, we overcome this limitation by showing that reconstructing features from models trained in a self-supervised manner is a sufficient training signal for object-centric representations to arise in a fully unsupervised way. Our approach, DINOSAUR, significantly out-performs existing image-based object-centric learning models on simulated data and is the first unsupervised object-centric model that scales to real-world datasets such as COCO and PASCAL VOC. DINOSAUR is conceptually simple and shows competitive performance compared to more involved pipelines from the computer vision literature.