mlsanityData to 14 Oct 2026

Towards Self-Supervised Learning of Global and Object-Centric Representations

arXiv 2203.05997 ↗First posted 11 Mar 2022cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Self-supervision allows learning meaningful representations of natural images, which usually contain one central object. How well does it transfer to multi-entity scenes? We discuss key aspects of learning structured object-centric representations with self-supervision and validate our insights through several experiments on the CLEVR dataset. Regarding the architecture, we confirm the importance of competition for attention-based object discovery, where each image patch is exclusively attended by one object. For training, we show that contrastive losses equipped with matching can be applied directly in a latent space, avoiding pixel-based reconstruction. However, such an optimization objective is sensitive to false negatives (recurring objects) and false positives (matching errors). Careful consideration is thus required around data augmentation and negative sample selection.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 37 of its 41 references that are in the map. First posted on arXiv 11 Mar 2022.

Builds on

37 of 41 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2020End-to-End Object Detection with Transformerscited by 7,414
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  11. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  12. 2013Auto-Encoding Variational Bayescited by 4,053
25 more
  1. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  2. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  3. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  4. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  5. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  6. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  7. 2016Colorful Image Colorizationcited by 1,114
  8. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  9. 2020Mastering Atari with Discrete World Modelscited by 1,007
  10. 2021Multiscale Vision Transformerscited by 897
  11. 2020Object-Centric Learning with Slot Attentioncited by 876
  12. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  13. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  14. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
  15. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 360
  16. 2021Conditional Object-Centric Learning from Videocited by 248
  17. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  18. 2019Contrastive Learning of Structured World Modelscited by 209
  19. 2021Efficient Self-supervised Vision Transformers for Representation Learningcited by 149
  20. 2019Unsupervised State Representation Learning in Ataricited by 144
  21. 2021How to represent part-whole hierarchies in a neural networkcited by 123
  22. 2017A Regularized Framework for Sparse and Structured Neural Attentioncited by 38
  23. 2020Learning Object-Centric Video Models by Contrasting Setscited by 19
  24. 2020Slot Contrastive Networks: A Contrastive Approach for Representing Objectscited by 9
  25. 2020Sparse and Structured Visual Attentioncited by 5

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 6; 2024: 1.

11 papers.

  1. 2023Object-Centric Slot Diffusioncited by 90
  2. 2023SlotDiffusion: Object-Centric Generative Modeling with Diffusion Modelscited by 85
  3. 2022Self-Supervised Visual Representation Learning with Semantic Groupingcited by 70
  4. 2023SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformerscited by 49
  5. 2022Neural Systematic Bindercited by 37
  6. 2023Rotating Features for Object Discoverycited by 29
  7. 2023Contrastive Training of Complex-Valued Autoencoders for Object Discoverycited by 16
  8. 2022Visual Concepts Tokenizationcited by 14
  9. 2022Robust and Controllable Object-Centric Learning through Energy-based Modelscited by 7
  10. 2024Binding Dynamics in Rotating Featurescited by 5
  11. 2023MOFO: MOtion FOcused Self-Supervision for Video Understandingcited by 3