mlsanityData to 14 Oct 2026

Unsupervised Semantic Segmentation by Distilling Feature Correspondences

arXiv 2203.08414 ↗First posted 16 Mar 2022cs.CV cs.AI cs.LGcited by 241

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Unsupervised semantic segmentation aims to discover and localize semantically meaningful categories within image corpora without any form of annotation. To solve this task, algorithms must produce features for every pixel that are both semantically meaningful and compact enough to form distinct clusters. Unlike previous works which achieve this with a single end-to-end framework, we propose to separate feature learning from cluster compactification. Empirically, we show that current unsupervised feature learning frameworks already generate dense features whose correlations are semantically consistent. This observation motivates us to design STEGO (Self-supervised Transformer with Energy-based Graph Optimization), a novel framework that distills unsupervised features into high-quality discrete semantic labels. At the core of STEGO is a novel contrastive loss function that encourages features to form compact clusters while preserving their relationships across the corpora. STEGO yields a significant improvement over the prior state of the art, on both the CocoStuff (+14 mIoU) and Cityscapes (+9 mIoU) semantic segmentation challenges.

Key facts. Cited by 241 papers in the map (data through 14 Oct 2026). Builds on 44 of its 69 references that are in the map. First posted on arXiv 16 Mar 2022.

Builds on

44 of 69 references.

References · 44

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  9. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  10. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  11. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  12. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
32 more
  1. 2017Non-local Neural Networkscited by 2,293
  2. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  3. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  4. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  5. 2021Vision Transformers for Dense Predictioncited by 1,803
  6. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  7. 2015Spatial Transformer Networkscited by 1,429
  8. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  9. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  10. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  11. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  12. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,028
  13. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  14. 2018Self-Attention Generative Adversarial Networkscited by 809
  15. 2014Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFscited by 554
  16. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  17. 2020SCAN: Learning to Classify Images without Labelscited by 352
  18. 2020Contrastive Clusteringcited by 265
  19. 2019Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relationscited by 247
  20. 2017graph2vec: Learning Distributed Representations of Graphscited by 230
  21. 2016Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Predictioncited by 217
  22. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  23. 2021PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clusteringcited by 163
  24. 2020Unsupervised Learning of Dense Visual Representationscited by 147
  25. 2019SegSort: Segmentation by Discriminative Sorting of Segmentscited by 96
  26. 2018Deep Feature Factorization For Concept Discoverycited by 64
  27. 2020Autoregressive Unsupervised Image Segmentationcited by 60
  28. 2015Learning visual groups from co-occurrences in space and timecited by 42
  29. 2018Convolutional CRFs for Semantic Segmentationcited by 18
  30. 2021Unsupervised Image Segmentation by Mutual Information Maximization and Adversarial Regularizationcited by 13
  31. 2021Weakly-Supervised Image Semantic Segmentation Using Graph Convolutional Networkscited by 8
  32. 2016Mini-Batch Spectral Clusteringcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 33; 2023: 79; 2024: 60; 2025: 42; 2026: 27.

241 papers.

Sort:

By area:

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  2. 2023Emergent Correspondence from Image Diffusioncited by 446
  3. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
  4. 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attentioncited by 236
  5. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 220
  6. 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondencecited by 214
  7. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 196
  8. 2023Segment Any 3D Gaussianscited by 161
  9. 2023Weakly Supervised 3D Open-vocabulary Segmentationcited by 135
  10. 2023Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusioncited by 131
  11. 2023Distilled Feature Fields Enable Few-Shot Language-Guided Manipulationcited by 130
  12. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
38 more
  1. 2022ReCo: Retrieve and Co-segment for Zero-shot Transfercited by 117
  2. 2026V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningcited by 117
  3. 2024FeatUp: A Model-Agnostic Framework for Features at Any Resolutioncited by 116
  4. 2022Open Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learningcited by 115
  5. 2024ClearCLIP: Decomposing CLIP Representations for Dense Vision-Language Inferencecited by 113
  6. 2024ProxyCLIP: Proxy Attention Improves CLIP for Open-Vocabulary Segmentationcited by 111
  7. 2023Hierarchical Dense Correlation Distillation for Few-Shot Segmentationcited by 84
  8. 2022Perceptual Grouping in Contrastive Vision-Language Modelscited by 82
  9. 2022NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenescited by 71
  10. 2022Self-Supervised Visual Representation Learning with Semantic Groupingcited by 70
  11. 2022LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discoverycited by 66
  12. 2022Unsupervised Object Localization: Observing the Background to Discover Objectscited by 66
  13. 2023LOCATE: Localize and Transfer Object Parts for Weakly Supervised Affordance Groundingcited by 64
  14. 2023Diffusion Models for Open-Vocabulary Segmentationcited by 64
  15. 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
  16. 2022Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentationcited by 55
  17. 2022Discovering Object Masks with Transformers for Unsupervised Semantic Segmentationcited by 54
  18. 2022Unsupervised Semantic Segmentation with Self-supervised Object-centric Representationscited by 54
  19. 2025ConceptAttention: Diffusion Transformers Learn Highly Interpretable Featurescited by 52
  20. 2022IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesiscited by 50
  21. 2023SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformerscited by 49
  22. 2022A Survey on Label-efficient Deep Image Segmentation: Bridging the Gap between Weak Supervision and Dense Predictioncited by 46
  23. 2023CoDet: Co-Occurrence Guided Region-Word Alignment for Open-Vocabulary Object Detectioncited by 46
  24. 2022Teaching Matters: Investigating the Role of Supervision in Vision Transformerscited by 45
  25. 2023Fast Traversability Estimation for Wild Visual Navigationcited by 44
  26. 2024EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Modelscited by 43
  27. 2022Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motioncited by 42
  28. 2022ACSeg: Adaptive Conceptualization for Unsupervised Semantic Segmentationcited by 42
  29. 2023Leveraging Hidden Positives for Unsupervised Semantic Segmentationcited by 40
  30. 2023VDD: Varied Drone Dataset for Semantic Segmentationcited by 40
  31. 2022Self-Guided Diffusion Modelscited by 37
  32. 2023Time Does Tell: Self-Supervised Time-Tuning of Dense Image Representationscited by 35
  33. 2023Unsupervised Universal Image Segmentationcited by 35
  34. 2024Open-Vocabulary Semantic Segmentation with Image Embedding Balancingcited by 35
  35. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  36. 2022Peekaboo: Text to Image Diffusion Models are Zero-Shot Segmentorscited by 34
  37. 2023CLIP-S4: Language-Guided Self-Supervised Semantic Segmentationcited by 34
  38. 2023Neural Congealing: Aligning Images to a Joint Semantic Atlascited by 31

50 most cited of 241 All 241 (JSON)