mlsanityData to 7 Oct 2026

Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videos

arXiv 2205.14065 ↗First posted 27 May 2022cs.CV cs.LGcited by 139

Not on the map. Why?

Abstract (arXiv)

Unsupervised object-centric learning aims to represent the modular, compositional, and causal structure of a scene as a set of object representations and thereby promises to resolve many critical limitations of traditional single-vector representations such as poor systematic generalization. Although there have been many remarkable advances in recent years, one of the most critical problems in this direction has been that previous methods work only with simple and synthetic scenes but not with complex and naturalistic images or videos. In this paper, we propose STEVE, an unsupervised model for object-centric learning in videos. Our proposed model makes a significant advancement by demonstrating its effectiveness on various complex and naturalistic videos unprecedented in this line of research. Interestingly, this is achieved by neither adding complexity to the model architecture nor introducing a new objective or weak supervision. Rather, it is achieved by a surprisingly simple architecture that uses a transformer-based image decoder conditioned on slots and the learning objective is simply to reconstruct the observation. Our experiment results on various complex and naturalistic videos show significant improvements compared to the previous state-of-the-art.

Key facts. Cited by 139 papers in the map (data through 7 Oct 2026). Builds on 73 of its 90 references that are in the map. First posted on arXiv 27 May 2022.

Builds on

73 of 90 references.

References · 73

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2013Describing Textures in the Wildcited by 2,110
  7. 2020Object-Centric Learning with Slot Attentioncited by 865
  8. 2019Video Object Segmentation using Space-Time Memory Networkscited by 436
  9. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  10. 2022Kubric: A scalable dataset generatorcited by 373
  11. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  12. 2019Learning Correspondence from the Cycle-Consistency of Timecited by 269
38 more
  1. 2021Conditional Object-Centric Learning from Videocited by 247
  2. 2020On the Binding Problem in Artificial Neural Networkscited by 230
  3. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  4. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  5. 2019Contrastive Learning of Structured World Modelscited by 206
  6. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  7. 2018Tracking Emerges by Colorizing Videoscited by 166
  8. 2022Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cutcited by 166
  9. 2021Self-supervised Video Object Segmentation by Motion Groupingcited by 147
  10. 2021Illiterate DALL-E Learns to Composecited by 147
  11. 2019CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoningcited by 137
  12. 2018Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objectscited by 128
  13. 2020What Matters in Unsupervised Optical Flowcited by 124
  14. 2017Neural Expectation Maximizationcited by 123
  15. 2019Entity Abstraction in Visual Model-Based Reinforcement Learningcited by 110
  16. 2020MAST: A Memory-Augmented Self-supervised Trackercited by 109
  17. 2021GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinementcited by 109
  18. 2019SCALOR: Generative World Models with Scalable Object Representationscited by 108
  19. 2017Occlusion Aware Unsupervised Learning of Optical Flowcited by 101
  20. 2018Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactionscited by 100
  21. 2021ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentationcited by 93
  22. 2021Decomposing 3D Scenes into Objects via Unsupervised Volume Segmentationcited by 88
  23. 2021Generalization and Robustness Implications in Object-Centric Learningcited by 81
  24. 2019Unsupervised Deep Trackingcited by 74
  25. 2021SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warpingcited by 74
  26. 2019Unsupervised Moving Object Detection via Contextual Information Separationcited by 72
  27. 2016It's Moving! A Probabilistic Model for Causal Motion Segmentation in Moving Camera Videoscited by 71
  28. 2021Unsupervised Learning of Compositional Energy Conceptscited by 71
  29. 2020Improving Generative Imagination in Object-Centric World Modelscited by 70
  30. 2016Tagger: Deep Unsupervised Perceptual Groupingcited by 69
  31. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
  32. 2020Generative Neurosymbolic Machinescited by 57
  33. 2019COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Explorationcited by 55
  34. 2019Structured agents for physical constructioncited by 50
  35. 2022Discovering Objects that Can Movecited by 44
  36. 2019Structured Object-Aware Physics Prediction for Video Modeling and Planningcited by 42
  37. 2019Exploiting Spatial Invariance for Scalable Unsupervised Object Trackingcited by 40
  38. 2020ROOTS: Object-Centric Representation and Rendering of 3D Scenescited by 40

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 40; 2024: 37; 2025: 31; 2026: 24.

139 papers.

Sort:

By area:

  1. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 219
  2. 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 146
  3. 2022Disentangled Representation Learningcited by 111
  4. 2023Object-Centric Slot Diffusioncited by 90
  5. 2023SlotDiffusion: Object-Centric Generative Modeling with Diffusion Modelscited by 85
  6. 2023Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similaritiescited by 78
  7. 2022Improving Object-centric Learning with Query Optimizationcited by 60
  8. 2022Unsupervised Semantic Segmentation with Self-supervised Object-centric Representationscited by 54
  9. 2023Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Propertiescited by 53
  10. 2023Invariant Slot Attention: Object Discovery with Slot-Centric Reference Framescited by 51
  11. 2023An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learningcited by 49
  12. 2023Self-supervised Object-Centric Learning for Videoscited by 49
38 more
  1. 2023SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformerscited by 49
  2. 2023Provably Learning Object-Centric Representationscited by 45
  3. 2022Neural Systematic Bindercited by 37
  4. 2024Artificial Kuramoto Oscillatory Neuronscited by 37
  5. 2024Adaptive Slot Attention: Object Discovery with Dynamic Slot Numbercited by 34
  6. 2023Helping Hands: An Object-Aware Ego-Centric Video Recognition Modelcited by 31
  7. 2023OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environmentscited by 30
  8. 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Modelscited by 30
  9. 2024Temporally Consistent Object-Centric Learning by Contrasting Slotscited by 30
  10. 2023Rotating Features for Object Discoverycited by 29
  11. 2023Object Discovery from Motion-Guided Tokenscited by 25
  12. 2026Causal-JEPA: Learning World Models through Object-Level Latent Maskingcited by 25
  13. 2026OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulationcited by 25
  14. 2023Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learningcited by 22
  15. 2023Systematic Visual Reasoning through Object-Centric Relational Abstractioncited by 19
  16. 2024Zero-Shot Object-Centric Representation Learningcited by 19
  17. 2024Interaction Asymmetry: A General Principle for Learning Composable Abstractionscited by 19
  18. 2025Does Object Binding Naturally Emerge in Large Pretrained Vision Transformers?cited by 19
  19. 2022Seeing the forest and the tree: Building representations of both individual and collective dynamics with transformerscited by 18
  20. 2023DDLP: Unsupervised Object-Centric Video Prediction with Deep Dynamic Latent Particlescited by 18
  21. 2023Shepherding Slots to Objects: Towards Stable and Robust Object-Centric Learningcited by 17
  22. 2024Slot State Space Modelscited by 17
  23. 2023Object-centric Learning with Cyclic Walks between Parts and Wholecited by 16
  24. 2025Object-Centric World Model for Language-Guided Manipulationcited by 16
  25. 2023Object-Centric Multiple Object Trackingcited by 15
  26. 2024Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Modelscited by 14
  27. 2024GLASS: Guided Latent Slot Diffusion for Object-Centric Learningcited by 14
  28. 2025Learning segmentation from point trajectoriescited by 14
  29. 2025Slot Attention with Re-Initialization and Self-Distillationcited by 14
  30. 2023Unsupervised Open-Vocabulary Object Localization in Videoscited by 13
  31. 2024Learning to Compose: Improving Object Centric Learning by Injecting Compositionalitycited by 13
  32. 2025Slot-Guided Adaptation of Pre-trained Diffusion Models for Object-Centric Learning and Compositional Generationcited by 13
  33. 2025PlaySlot: Learning Inverse Latent Dynamics for Controllable Object-Centric Video Prediction and Planningcited by 13
  34. 2024SlotLifter: Slot-guided Feature Lifting for Learning Object-centric Radiance Fieldscited by 12
  35. 2025Predicting Video Slot Attention Queries from Random Slot-Feature Pairscited by 12
  36. 2025Vector-Quantized Vision Foundation Models for Object-Centric Learningcited by 11
  37. 2025MetaSlot: Break Through the Fixed Number of Slots in Object-Centric Learningcited by 11
  38. 2026Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modelingcited by 11

50 most cited of 139 All 139 (JSON)