mlsanityData to 7 Oct 2026

Generalization and Robustness Implications in Object-Centric Learning

arXiv 2107.00637 ↗First posted 1 Jul 2021cs.LG cs.CV stat.MLcited by 81

Not on the map. Why?

Abstract (arXiv)

The idea behind object-centric representation learning is that natural scenes can better be modeled as compositions of objects and their relations as opposed to distributed representations. This inductive bias can be injected into neural networks to potentially improve systematic generalization and performance of downstream tasks in scenes with multiple objects. In this paper, we train state-of-the-art unsupervised models on five common multi-object datasets and evaluate segmentation metrics and downstream object property prediction. In addition, we study generalization and robustness by investigating the settings where either a single object is out of distribution -- e.g., having an unseen color, texture, or shape -- or global properties of the scene are altered -- e.g., by occlusions, cropping, or increasing the number of objects. From our experimental study, we find object-centric representations to be useful for downstream tasks and generally robust to most distribution shifts affecting objects. However, when the distribution shift affects the input in a less structured manner, robustness in terms of segmentation and downstream task performance may vary significantly across models and distribution shifts.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). Builds on 53 of its 112 references that are in the map. First posted on arXiv 1 Jul 2021.

Builds on

53 of 112 references.

References · 53

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  5. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  6. 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
  7. 2020Object-Centric Learning with Slot Attentioncited by 865
  8. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 660
  9. 2018Disentangling by Factorisingcited by 614
  10. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 503
  11. 2015A Neural Algorithm of Artistic Stylecited by 480
  12. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
38 more
  1. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  2. 2014Recurrent Models of Visual Attentioncited by 344
  3. 2019Causality for Machine Learningcited by 242
  4. 2017Variational Inference of Disentangled Latent Concepts from Unlabeled Observationscited by 240
  5. 2020On the Binding Problem in Artificial Neural Networkscited by 230
  6. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  7. 2019Recurrent Independent Mechanismscited by 219
  8. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  9. 2021Towards Causal Representation Learningcited by 207
  10. 2019Contrastive Learning of Structured World Modelscited by 206
  11. 2015DRAW: A Recurrent Neural Network For Image Generationcited by 190
  12. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  13. 2018Isolating Sources of Disentanglement in Variational Autoencoderscited by 167
  14. 2019On the Fairness of Disentangled Representationscited by 144
  15. 2018Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objectscited by 128
  16. 2017Neural Expectation Maximizationcited by 123
  17. 2019Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEscited by 116
  18. 2021GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinementcited by 109
  19. 2019SCALOR: Generative World Models with Scalable Object Representationscited by 108
  20. 2018Learning Deep Disentangled Embeddings with the F-Statistic Losscited by 101
  21. 2018Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactionscited by 100
  22. 2019Are Disentangled Representations Helpful for Abstract Visual Reasoning?cited by 100
  23. 2019On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Datasetcited by 100
  24. 2017Building machines that adapt and compute like brainscited by 99
  25. 2020On Disentangled Representations Learned From Correlated Datacited by 94
  26. 2019Unsupervised Object Segmentation by Redrawingcited by 74
  27. 2020Improving Generative Imagination in Object-Centric World Modelscited by 70
  28. 2016Tagger: Deep Unsupervised Perceptual Groupingcited by 69
  29. 2020On the Transfer of Disentangled Representations in Realistic Settingscited by 63
  30. 2018Structured Disentangled Representationscited by 62
  31. 2018ShapeStacks: Learning Vision-Based Physical Intuition for Generalised Object Stackingcited by 55
  32. 2020A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluationcited by 52
  33. 2019Structured Object-Aware Physics Prediction for Video Modeling and Planningcited by 42
  34. 2019Exploiting Spatial Invariance for Scalable Unsupervised Object Trackingcited by 40
  35. 2020Towards causal generative scene models via competition of expertscited by 37
  36. 2020Learning to Manipulate Individual Objects in an Imagecited by 22
  37. 2020Learning Object-Centric Video Models by Contrasting Setscited by 19
  38. 2020Reconstruction Bottlenecks in Object-Centric Generative Modelscited by 17

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 13; 2023: 20; 2024: 23; 2025: 13; 2026: 9.

81 papers.

Sort:

By area:

  1. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 219
  2. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  3. 2023Object-Centric Slot Diffusioncited by 90
  4. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 56
  5. 2022Assaying Out-Of-Distribution Generalization in Transfer Learningcited by 53
  6. 2023An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learningcited by 49
  7. 2023Self-supervised Object-Centric Learning for Videoscited by 49
  8. 2023Provably Learning Object-Centric Representationscited by 45
  9. 2022MoCoDA: Model-based Counterfactual Data Augmentationcited by 44
  10. 2023Provable Compositional Generalization for Object-Centric Learningcited by 41
  11. 2022Complex-Valued Autoencoders for Object Discoverycited by 40
  12. 2022Neural Systematic Bindercited by 37
38 more
  1. 2024Adaptive Slot Attention: Object Discovery with Dynamic Slot Numbercited by 34
  2. 2023OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environmentscited by 30
  3. 2024What is the Visual Cognition Gap between Humans and Multimodal LLMs?cited by 30
  4. 2024Temporally Consistent Object-Centric Learning by Contrasting Slotscited by 30
  5. 2022Compositional Scene Representation Learning via Reconstruction: A Surveycited by 27
  6. 2023Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learningcited by 22
  7. 2023Systematic Visual Reasoning through Object-Centric Relational Abstractioncited by 19
  8. 2024Zero-Shot Object-Centric Representation Learningcited by 19
  9. 2024Interaction Asymmetry: A General Principle for Learning Composable Abstractionscited by 19
  10. 2023FOCUS: Object-Centric World Models for Robotics Manipulationcited by 17
  11. 2021Neuro-Symbolic Forward Reasoningcited by 16
  12. 2024Composing Pre-Trained Object-Centric Representations for Robotics From "What" and "Where" Foundation Modelscited by 16
  13. 2023Object-Centric Multiple Object Trackingcited by 15
  14. 2024Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Modelscited by 14
  15. 2024GLASS: Guided Latent Slot Diffusion for Object-Centric Learningcited by 14
  16. 2022Inductive Biases for Object-Centric Representations in the Presence of Complex Texturescited by 13
  17. 2023Unsupervised Open-Vocabulary Object Localization in Videoscited by 13
  18. 2024Learning to Compose: Improving Object Centric Learning by Injecting Compositionalitycited by 13
  19. 2025PlaySlot: Learning Inverse Latent Dynamics for Controllable Object-Centric Video Prediction and Planningcited by 13
  20. 2021The Role of Pretrained Representations for the OOD Generalization of Reinforcement Learning Agentscited by 12
  21. 2024Slot Abstractors: Toward Scalable Abstract Visual Reasoningcited by 10
  22. 2025Are We Done with Object-Centric Learning?cited by 10
  23. 2024Neural Language of Thought Modelscited by 9
  24. 2025Object-Centric Latent Action Learningcited by 9
  25. 2022A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunitiescited by 8
  26. 2023Sensitivity of Slot-Based Object-Centric Models to their Number of Slotscited by 8
  27. 2023Spotlight Attention: Robust Object-Centric Learning With a Spatial Locality Priorcited by 8
  28. 2022Robust and Controllable Object-Centric Learning through Energy-based Modelscited by 7
  29. 2023Cycle Consistency Driven Object Discoverycited by 7
  30. 2024Reasoning-Enhanced Object-Centric Learning for Videoscited by 7
  31. 2022Differentiable Mathematical Programming for Object-Centric Representation Learningcited by 6
  32. 2023RobustCLEVR: A Benchmark and Framework for Evaluating Robustness in Object-centric Learningcited by 6
  33. 2023Imagine the Unseen World: A Benchmark for Systematic Generalization in Visual World Modelscited by 6
  34. 2024Recasting Generic Pretrained Vision Transformers As Object-Centric Scene Encoders For Manipulation Policiescited by 6
  35. 2024Bootstrapping Top-down Information for Self-modulating Slot Attentioncited by 6
  36. 2024Binding Dynamics in Rotating Featurescited by 5
  37. 2024Towards the Reusability and Compositionality of Causal Representationscited by 5
  38. 2024Object-Centric Temporal Consistency via Conditional Autoregressive Inductive Biasescited by 5

50 most cited of 81 All 81 (JSON)