mlsanityData to 7 Oct 2026

Unsupervised Semantic Segmentation with Self-supervised Object-centric Representations

arXiv 2207.05027 ↗First posted 11 Jul 2022cs.CVcited by 54

Not on the map. Why?

Abstract (arXiv)

In this paper, we show that recent advances in self-supervised feature learning enable unsupervised object discovery and semantic segmentation with a performance that matches the state of the field on supervised semantic segmentation 10 years ago. We propose a methodology based on unsupervised saliency masks and self-supervised feature clustering to kickstart object discovery followed by training a semantic segmentation network on pseudo-labels to bootstrap the system on images with multiple objects. We present results on PASCAL VOC that go far beyond the current state of the art (50.0 mIoU), and we report for the first time results on MS COCO for the whole set of 81 classes: our method discovers 34 categories with more than 20% IoU, while obtaining an average IoU of 19.6 for all 81 categories.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 62 of its 77 references that are in the map. First posted on arXiv 11 Jul 2022.

Builds on

62 of 77 references.

References · 62

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  8. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  10. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  11. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  12. 2016Colorful Image Colorizationcited by 1,113
38 more
  1. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  2. 2020Object-Centric Learning with Slot Attentioncited by 865
  3. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
  4. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 454
  5. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  6. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  7. 2020SCAN: Learning to Classify Images without Labelscited by 351
  8. 2016ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentationcited by 330
  9. 2015What's the Point: Semantic Segmentation with Point Supervisioncited by 297
  10. 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
  11. 2016Segmentation from Natural Language Expressionscited by 257
  12. 2021Conditional Object-Centric Learning from Videocited by 247
  13. 2022Unsupervised Semantic Segmentation by Distilling Feature Correspondencescited by 241
  14. 2015BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentationcited by 238
  15. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  16. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 227
  17. 2016Simple Does It: Weakly Supervised Instance and Semantic Segmentationcited by 226
  18. 2016Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentationcited by 198
  19. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  20. 2020Single-Stage Semantic Segmentation from Image Labelscited by 167
  21. 2022Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cutcited by 166
  22. 2021PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clusteringcited by 163
  23. 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
  24. 2019Semi-Supervised Semantic Segmentation with High- and Low-level Consistencycited by 147
  25. 2021Illiterate DALL-E Learns to Composecited by 147
  26. 2018Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentationcited by 145
  27. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  28. 2017Deep Extreme Cut: From Extreme Points to Object Segmentationcited by 127
  29. 2021Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentationcited by 127
  30. 2018Learning to Decompose and Disentangle Representations for Video Predictioncited by 120
  31. 2019SegSort: Segmentation by Discriminative Sorting of Segmentscited by 96
  32. 2022Self-Supervised Learning of Object Parts for Semantic Segmentationcited by 90
  33. 2014Salient Object Detection: A Discriminative Regional Feature Integration Approachcited by 89
  34. 2018Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Featurescited by 81
  35. 2017Extreme clicking for efficient object annotationcited by 80
  36. 2022Object discovery and representation networkscited by 80
  37. 2021Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentationcited by 77
  38. 2020Toward unsupervised, multi-object discovery in large-scale image collectionscited by 65

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 19; 2024: 21; 2025: 6; 2026: 2.

54 papers.

Sort:

By area:

  1. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 219
  2. 2022Improving Sample Quality of Diffusion Models Using Self-Attention Guidancecited by 151
  3. 2023Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusioncited by 131
  4. 2022Perceptual Grouping in Contrastive Vision-Language Modelscited by 82
  5. 2022Unsupervised Object Localization: Observing the Background to Discover Objectscited by 66
  6. 2022ACSeg: Adaptive Conceptualization for Unsupervised Semantic Segmentationcited by 42
  7. 2024EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Modelscited by 42
  8. 2023CLIP-DIY: CLIP Dense Inference Yields Open-Vocabulary Semantic Segmentation For-Freecited by 39
  9. 2024In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic Segmentationcited by 38
  10. 2023Time Does Tell: Self-Supervised Time-Tuning of Dense Image Representationscited by 35
  11. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  12. 2024Adaptive Slot Attention: Object Discovery with Dynamic Slot Numbercited by 34
38 more
  1. 2023Rotating Features for Object Discoverycited by 29
  2. 2024EAGLE: Eigen Aggregation Learning for Object-Centric Unsupervised Semantic Segmentationcited by 27
  3. 2023Distilling Self-Supervised Vision Transformers for Weakly-Supervised Few-Shot Classification & Segmentationcited by 24
  4. 2024Zero-Shot Object-Centric Representation Learningcited by 19
  5. 2023Causal Unsupervised Semantic Segmentationcited by 18
  6. 2023CrIBo: Self-Supervised Learning via Cross-Image Object-Level Bootstrappingcited by 17
  7. 2023Object-centric Learning with Cyclic Walks between Parts and Wholecited by 16
  8. 2022Far Away in the Deep Space: Dense Nearest-Neighbor-Based Out-of-Distribution Detectioncited by 15
  9. 2024Near, far: Patch-ordering enhances vision foundation models' scene understandingcited by 15
  10. 2024GLASS: Guided Latent Slot Diffusion for Object-Centric Learningcited by 14
  11. 2023AutoRecon: Automated 3D Object Discovery and Reconstructioncited by 13
  12. 2023Unsupervised Open-Vocabulary Object Localization in Videoscited by 13
  13. 2023Weakly-supervised Contrastive Learning for Unsupervised Object Discoverycited by 12
  14. 2023Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentationcited by 10
  15. 2024Boosting Unsupervised Semantic Segmentation with Principal Mask Proposalscited by 9
  16. 2023LOCATE: Self-supervised Object Discovery via Flow-guided Graph-cut and Bootstrapped Self-trainingcited by 8
  17. 2024Rethinking Image-to-Video Adaptation: An Object-centric Perspectivecited by 8
  18. 2025LogoSP: Local-global Grouping of Superpoints for Unsupervised Semantic Segmentation of 3D Point Cloudscited by 8
  19. 2024Exploring Simple Open-Vocabulary Semantic Segmentationcited by 6
  20. 2023Learning to Count without Annotationscited by 5
  21. 2025MoSiC: Optimal-Transport Motion Trajectory for Dense Self-Supervised Learningcited by 5
  22. 2023Open Vocabulary Semantic Scene Sketch Understandingcited by 4
  23. 2024SPARO: Selective Attention for Robust and Compositional Transformer Encodings for Visioncited by 4
  24. 2024Masked Multi-Query Slot Attention for Unsupervised Object Discoverycited by 3
  25. 2023SimCon Loss with Multiple Views for Text Supervised Semantic Segmentationcited by 2
  26. 2023Generalized 3D Self-supervised Learning Framework via Prompted Foreground-Aware Feature Contrastcited by 2
  27. 2024Applying Unsupervised Semantic Segmentation to High-Resolution UAV Imagery for Enhanced Road Scene Parsingcited by 2
  28. 2024SimSAM: Simple Siamese Representations Based Semantic Affinity Matrix for Unsupervised Image Segmentationcited by 2
  29. 2025FlowCut: Unsupervised Video Instance Segmentation via Temporal Mask Matchingcited by 2
  30. 2023A Lightweight Clustering Framework for Unsupervised Semantic Segmentationcited by 1
  31. 2024Deep Spectral Improvement for Unsupervised Image Instance Segmentationcited by 1
  32. 2023Repurposing SAM for User-Defined Semantics Aware Segmentationcited by 0
  33. 2024Semi-supervised segmentation of land cover images using nonlinear canonical correlation analysis with multiple features and t-SNEcited by 0
  34. 2024OMH: Structured Sparsity via Optimally Matched Hierarchy for Unsupervised Semantic Segmentationcited by 0
  35. 2024Joint-Embedding Predictive Architecture for Self-Supervised Learning of Mask Classification Architecturecited by 0
  36. 2024PEEKABOO: Hiding parts of an image for unsupervised object localizationcited by 0
  37. 2024Improving Weakly-supervised Video Instance Segmentation by Leveraging Spatio-temporal Consistencycited by 0
  38. 2025Component Adaptive Clustering for Generalized Category Discoverycited by 0

50 most cited of 54 All 54 (JSON)