# SCALOR: Generative World Models with Scalable Object Representations

arXiv: https://arxiv.org/abs/1910.02384 · HTML: https://mlsanity.com/paper/1910.02384/

## Key facts

- Cited by 108 papers in the map (data through 14 Oct 2026).
- Placed in Latent predictive world models, one of 11 directions in World models.
- Builds on 0 of its 26 references that are in the map.
- First posted on arXiv 6 Oct 2019.

## Placed in

- World models / [Latent predictive world models](https://mlsanity.com/wm/latent-predictive-world-models/)

## Built on by (most cited first, 30 of 108)

- [Conditional Object-Centric Learning from Video](https://mlsanity.com/paper/2111.12594/) (2021)
- [Bridging the Gap to Real-World Object-Centric Learning](https://mlsanity.com/paper/2209.14860/) (2022)
- [SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videos](https://mlsanity.com/paper/2206.07764/) (2022)
- [Illiterate DALL-E Learns to Compose](https://mlsanity.com/paper/2110.11405/) (2021)
- [SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models](https://mlsanity.com/paper/2210.05861/) (2022)
- [Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videos](https://mlsanity.com/paper/2205.14065/) (2022)
- [GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement](https://mlsanity.com/paper/2104.09958/) (2021)
- [Object-Centric Slot Diffusion](https://mlsanity.com/paper/2303.10834/) (2023)
- [SlotDiffusion: Object-Centric Generative Modeling with Diffusion Models](https://mlsanity.com/paper/2305.11281/) (2023)
- [Generalization and Robustness Implications in Object-Centric Learning](https://mlsanity.com/paper/2107.00637/) (2021)
- [Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similarities](https://mlsanity.com/paper/2306.04829/) (2023)
- [SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decomposition](https://mlsanity.com/paper/2106.03849/) (2021)
- [Improving Object-centric Learning with Query Optimization](https://mlsanity.com/paper/2210.08990/) (2022)
- [Unsupervised Part Discovery from Contrastive Reconstruction](https://mlsanity.com/paper/2111.06349/) (2021)
- [Invariant Slot Attention: Object Discovery with Slot-Centric Reference Frames](https://mlsanity.com/paper/2302.04973/) (2023)
- [Opening up Open-World Tracking](https://mlsanity.com/paper/2104.11221/) (2021)
- [An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learning](https://mlsanity.com/paper/2302.04419/) (2023)
- [Self-supervised Object-Centric Learning for Videos](https://mlsanity.com/paper/2310.06907/) (2023)
- [Discovering Objects that Can Move](https://mlsanity.com/paper/2203.10159/) (2022)
- [MoCoDA: Model-based Counterfactual Data Augmentation](https://mlsanity.com/paper/2210.11287/) (2022)
- [Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motion](https://mlsanity.com/paper/2205.07844/) (2022)
- [Neural Systematic Binder](https://mlsanity.com/paper/2211.01177/) (2022)
- [MarioNette: Self-Supervised Sprite Learning](https://mlsanity.com/paper/2104.14553/) (2021)
- [From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence](https://mlsanity.com/paper/2110.15245/) (2021)
- [VAEL: Bridging Variational Autoencoders and Probabilistic Logic Programming](https://mlsanity.com/paper/2202.04178/) (2022)
- [Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulation](https://mlsanity.com/paper/2206.11403/) (2022)
- [Temporally Consistent Object-Centric Learning by Contrasting Slots](https://mlsanity.com/paper/2412.14295/) (2024)
- [Unsupervised Object Learning via Common Fate](https://mlsanity.com/paper/2110.06562/) (2021)
- [Compositional Scene Representation Learning via Reconstruction: A Survey](https://mlsanity.com/paper/2202.07135/) (2022)
- [Entity-Centric Reinforcement Learning for Object Manipulation from Pixels](https://mlsanity.com/paper/2404.01220/) (2024)

## Abstract (arXiv)

Scalability in terms of object density in a scene is a primary challenge in unsupervised sequential object-oriented representation learning. Most of the previous models have been shown to work only on scenes with a few objects. In this paper, we propose SCALOR, a probabilistic generative world model for learning SCALable Object-oriented Representation of a video. With the proposed spatially-parallel attention and proposal-rejection mechanisms, SCALOR can deal with orders of magnitude larger numbers of objects compared to the previous state-of-the-art models. Additionally, we introduce a background module that allows SCALOR to model complex dynamic backgrounds as well as many foreground objects in the scene. We demonstrate that SCALOR can deal with crowded scenes containing up to a hundred objects while jointly modeling complex dynamic backgrounds. Importantly, SCALOR is the first unsupervised object representation model shown to work for natural scenes containing several tens of moving objects.