mlsanityData to 14 Oct 2026

Boosting Unsupervised Semantic Segmentation with Principal Mask Proposals

arXiv 2404.16818 ↗First posted 25 Apr 2024cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Unsupervised semantic segmentation aims to automatically partition images into semantically meaningful regions by identifying global semantic categories within an image corpus without any form of annotation. Building upon recent advances in self-supervised representation learning, we focus on how to leverage these large pre-trained models for the downstream task of unsupervised segmentation. We present PriMaPs - Principal Mask Proposals - decomposing images into semantically meaningful masks based on their feature representation. This allows us to realize unsupervised semantic segmentation by fitting class prototypes to PriMaPs with a stochastic expectation-maximization algorithm, PriMaPs-EM. Despite its conceptual simplicity, PriMaPs-EM leads to competitive results across various pre-trained backbone models, including DINO and DINOv2, and across different datasets, such as Cityscapes, COCO-Stuff, and Potsdam-3. Importantly, PriMaPs-EM is able to boost results when applied orthogonally to current state-of-the-art unsupervised semantic segmentation pipelines. Code is available at https://github.com/visinf/primaps.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 48 of its 64 references that are in the map. First posted on arXiv 25 Apr 2024.

Builds on

48 of 64 references.

References · 48

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2023Segment Anythingcited by 8,234
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  11. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  12. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
36 more
  1. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  3. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  4. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  5. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  6. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  7. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  8. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  9. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,028
  10. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  11. 2020Image Segmentation Using Deep Learning: A Surveycited by 660
  12. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  13. 2021Intriguing Properties of Vision Transformerscited by 446
  14. 2021Deep ViT Features as Dense Visual Descriptorscited by 330
  15. 2022Unsupervised Semantic Segmentation by Distilling Feature Correspondencescited by 241
  16. 2015BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentationcited by 238
  17. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 220
  18. 2020How Well Do Self-Supervised Models Transfer?cited by 213
  19. 2022Splicing ViT Features for Semantic Appearance Transfercited by 209
  20. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  21. 2023Cut and Learn for Unsupervised Object Detection and Instance Segmentationcited by 187
  22. 2020Single-Stage Semantic Segmentation from Image Labelscited by 167
  23. 2021PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clusteringcited by 163
  24. 2022Multi-class Token Transformer for Weakly Supervised Semantic Segmentationcited by 138
  25. 2022TokenCut: Segmenting Objects in Images and Videos with Self-supervised Transformer and Normalized Cutcited by 113
  26. 2019SegSort: Segmentation by Discriminative Sorting of Segmentscited by 96
  27. 2023Token Contrast for Weakly-Supervised Semantic Segmentationcited by 65
  28. 2022Discovering Object Masks with Transformers for Unsupervised Semantic Segmentationcited by 54
  29. 2022Unsupervised Semantic Segmentation with Self-supervised Object-centric Representationscited by 54
  30. 2015Learning visual groups from co-occurrences in space and timecited by 42
  31. 2021Background-Aware Pooling and Noise-Aware Loss for Weakly-Supervised Semantic Segmentationcited by 41
  32. 2023Leveraging Hidden Positives for Unsupervised Semantic Segmentationcited by 40
  33. 2022Unsupervised Hierarchical Semantic Segmentation with Multiview Cosegmentation and Clustering Transformerscited by 39
  34. 2021TransFGU: A Top-down Approach to Fine-Grained Unsupervised Semantic Segmentationcited by 38
  35. 2023Uncovering the Inner Workings of STEGO for Safe Unsupervised Semantic Segmentationcited by 7
  36. 2021ViCE: Improving Dense Representation Learning by Superpixelization and Contrasting Cluster Assignmentcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 3.

9 papers.

  1. 2025Feed-Forward SceneDINO for Unsupervised Semantic Scene Completioncited by 13
  2. 2025Scene-Centric Unsupervised Panoptic Segmentationcited by 10
  3. 2024Rethinking Decoders for Transformer-based Semantic Segmentation: A Compression Perspectivecited by 3
  4. 2025WakeupUrban: Unsupervised Semantic Segmentation of Mid-20th century Urban Landscapes with Satellite Imagerycited by 1
  5. 2026SegFly: A Dataset and 2D-3D-2D Paradigm for Aerial RGB-Thermal Semantic Segmentation at Scalecited by 1
  6. 2025Federated Unsupervised Semantic Segmentationcited by 0
  7. 2025MSSSeg: Learning Multi-Scale Structural Complexity for Self-Supervised Segmentationcited by 0
  8. 2026Unsupervised Semantic Segmentation Facilitates Model Understandingcited by 0
  9. 2026CrevasseSeg: A Label-Efficient UAV Crevasse Segmentation Frameworkcited by 0