mlsanityData to 14 Oct 2026

SCALOR: Generative World Models with Scalable Object Representations

arXiv 1910.02384 ↗First posted 6 Oct 2019cs.LG stat.MLcited by 108

On the map: World models → Latent predictive world models

Abstract (arXiv)

Scalability in terms of object density in a scene is a primary challenge in unsupervised sequential object-oriented representation learning. Most of the previous models have been shown to work only on scenes with a few objects. In this paper, we propose SCALOR, a probabilistic generative world model for learning SCALable Object-oriented Representation of a video. With the proposed spatially-parallel attention and proposal-rejection mechanisms, SCALOR can deal with orders of magnitude larger numbers of objects compared to the previous state-of-the-art models. Additionally, we introduce a background module that allows SCALOR to model complex dynamic backgrounds as well as many foreground objects in the scene. We demonstrate that SCALOR can deal with crowded scenes containing up to a hundred objects while jointly modeling complex dynamic backgrounds. Importantly, SCALOR is the first unsupervised object representation model shown to work for natural scenes containing several tens of moving objects.

Key facts. Cited by 108 papers in the map (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 0 of its 26 references that are in the map. First posted on arXiv 6 Oct 2019.

Builds on

0 of 26 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 22; 2022: 25; 2023: 24; 2024: 19; 2025: 10; 2026: 8.

108 papers.

Sort:

By area:

  1. 2021Conditional Object-Centric Learning from Videocited by 248
  2. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 220
  3. 2022SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videoscited by 166
  4. 2021Illiterate DALL-E Learns to Composecited by 147
  5. 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 147
  6. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  7. 2021GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinementcited by 109
  8. 2023Object-Centric Slot Diffusioncited by 90
  9. 2023SlotDiffusion: Object-Centric Generative Modeling with Diffusion Modelscited by 85
  10. 2021Generalization and Robustness Implications in Object-Centric Learningcited by 81
  11. 2023Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similaritiescited by 78
  12. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
38 more
  1. 2022Improving Object-centric Learning with Query Optimizationcited by 60
  2. 2021Unsupervised Part Discovery from Contrastive Reconstructioncited by 57
  3. 2023Invariant Slot Attention: Object Discovery with Slot-Centric Reference Framescited by 51
  4. 2021Opening up Open-World Trackingcited by 50
  5. 2023An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learningcited by 49
  6. 2023Self-supervised Object-Centric Learning for Videoscited by 49
  7. 2022Discovering Objects that Can Movecited by 44
  8. 2022MoCoDA: Model-based Counterfactual Data Augmentationcited by 44
  9. 2022Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motioncited by 42
  10. 2022Neural Systematic Bindercited by 37
  11. 2021MarioNette: Self-Supervised Sprite Learningcited by 35
  12. 2021From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligencecited by 35
  13. 2022VAEL: Bridging Variational Autoencoders and Probabilistic Logic Programmingcited by 31
  14. 2022Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulationcited by 31
  15. 2024Temporally Consistent Object-Centric Learning by Contrasting Slotscited by 31
  16. 2021Unsupervised Object Learning via Common Fatecited by 29
  17. 2022Compositional Scene Representation Learning via Reconstruction: A Surveycited by 27
  18. 2024Entity-Centric Reinforcement Learning for Object Manipulation from Pixelscited by 27
  19. 2022OGC: Unsupervised 3D Object Segmentation from Rigid Dynamics of Point Cloudscited by 25
  20. 2023Object Discovery from Motion-Guided Tokenscited by 25
  21. 2021Generative Video Transformer: Can Objects be the Words?cited by 24
  22. 2023Towards Unsupervised Object Detection From LiDAR Point Cloudscited by 23
  23. 2021Structured World Belief for Reinforcement Learning in POMDPcited by 21
  24. 2023Slot-VAE: Object-Centric Scene Generation with Slot Attentioncited by 20
  25. 2022Unsupervised Multi-object Segmentation by Predicting Probable Motion Patternscited by 19
  26. 2023Object-Centric Video Prediction via Decoupling of Object Dynamics and Interactionscited by 19
  27. 2023DDLP: Unsupervised Object-Centric Video Prediction with Deep Dynamic Latent Particlescited by 19
  28. 2022Boosting Object Representation Learning via Motion and Object Continuitycited by 17
  29. 2023Shepherding Slots to Objects: Towards Stable and Robust Object-Centric Learningcited by 17
  30. 2024Slot State Space Modelscited by 17
  31. 2021Self-supervised Reinforcement Learning with Independently Controllable Subgoalscited by 16
  32. 2023Multi-Object Discovery by Low-Dimensional Object Motioncited by 16
  33. 2023Object-Centric Multiple Object Trackingcited by 15
  34. 2026Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modelingcited by 15
  35. 2022Motion-inductive Self-supervised Object Discovery in Videoscited by 14
  36. 2022Self-supervised Amodal Video Object Segmentationcited by 14
  37. 2024Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Modelscited by 14
  38. 2021Reasoning-Modulated Representationscited by 13

50 most cited of 108 All 108 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens
  2. 2026MemLearner: Learning to Query Context memory for Video World Models
  3. 2026Physical Object Understanding with a Physically Controllable World Model
  4. 2026MOSH-WM: Mask-Grounded Soft-Hamiltonian Dynamics for Object-Centric World Models
  5. 2024ObjectMate: A Recurrence Prior for Object Insertion and Subject-Driven Generation
  6. 2025Compositional Video Synthesis by Temporal Object-Centric Learning
  7. 2023Learning Object State Changes in Videos: An Open-World Perspective
  8. 2025DIP: Unsupervised Dense In-Context Post-training of Visual Representations