mlsanityData to 7 Oct 2026

PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering

arXiv 2103.17070 ↗First posted 30 Mar 2021cs.CVcited by 163

Not on the map. Why?

Abstract (arXiv)

We present a new framework for semantic segmentation without annotations via clustering. Off-the-shelf clustering methods are limited to curated, single-label, and object-centric images yet real-world data are dominantly uncurated, multi-label, and scene-centric. We extend clustering from images to pixels and assign separate cluster membership to different instances within each image. However, solely relying on pixel-wise feature similarity fails to learn high-level semantic concepts and overfits to low-level visual cues. We propose a method to incorporate geometric consistency as an inductive bias to learn invariance and equivariance for photometric and geometric variations. With our novel learning objective, our framework can learn high-level semantic concepts. Our method, PiCIE (Pixel-level feature Clustering using Invariance and Equivariance), is the first method capable of segmenting both things and stuff categories without any hyperparameter tuning or task-specific pre-processing. Our method largely outperforms existing baselines on COCO and Cityscapes with +17.5 Acc. and +4.5 mIoU. We show that PiCIE gives a better initialization for standard supervised training. The code is available at https://github.com/janghyuncho/PiCIE.

Key facts. Cited by 163 papers in the map (data through 7 Oct 2026). Builds on 46 of its 67 references that are in the map. First posted on arXiv 30 Mar 2021.

Builds on

46 of 67 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  9. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  10. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  11. 2017Billion-scale similarity search with GPUscited by 2,477
  12. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
34 more
  1. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  2. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  3. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  4. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  5. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 763
  6. 2019Panoptic Feature Pyramid Networkscited by 644
  7. 2016Fully-Convolutional Siamese Networks for Object Trackingcited by 566
  8. 2019Self-labelling via simultaneous clustering and representation learningcited by 525
  9. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 454
  10. 2020SCAN: Learning to Classify Images without Labelscited by 351
  11. 2019Learning to Discover Novel Visual Categories via Deep Transfer Clusteringcited by 223
  12. 2016Joint Unsupervised Learning of Deep Representations and Image Clusterscited by 205
  13. 2019Local Aggregation for Unsupervised Learning of Visual Embeddingscited by 204
  14. 2020Automatically Discovering and Learning New Visual Categories with Ranking Statisticscited by 141
  15. 2018Unsupervised Learning of Object Landmarks through Conditional Image Generationcited by 113
  16. 2019Unsupervised Learning of Object Keypoints for Perception and Controlcited by 109
  17. 2020Online Deep Clustering for Unsupervised Representation Learningcited by 99
  18. 2019Unsupervised Learning of Object Structure and Dynamics from Videoscited by 92
  19. 2019PREDICT & CLUSTER: Unsupervised Skeleton Based Action Recognitioncited by 90
  20. 2019ClusterFit: Improving Generalization of Visual Representationscited by 82
  21. 2020AD-Cluster: Augmented Discriminative Clustering for Domain Adaptive Person Re-identificationcited by 81
  22. 2018Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoningcited by 79
  23. 2019Unsupervised Object Segmentation by Redrawingcited by 74
  24. 2018Unsupervised Discovery of Object Landmarks as Structural Representationscited by 73
  25. 2019Unsupervised Part-Based Disentangling of Object Shape and Appearancecited by 67
  26. 2017Unsupervised learning of object landmarks by factorized spatial embeddingscited by 64
  27. 2019Emergence of Object Segmentation in Perturbed Generative Modelscited by 57
  28. 2018Self-supervised learning of a facial attribute embedding from videocited by 44
  29. 2015Learning visual groups from co-occurrences in space and timecited by 42
  30. 2019Self-Supervised Convolutional Subspace Clustering Networkcited by 32
  31. 2018Self-supervised Learning of Geometrically Stable Features Through Probabilistic Introspectioncited by 27
  32. 2020Unsupervised Deep Metric Learning with Transformed Attention Consistency and Contrastive Clustering Losscited by 18
  33. 2020Deep Image Clustering with Category-Style Representationcited by 10
  34. 2020Learning to Cluster under Domain Shiftcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 36; 2023: 55; 2024: 37; 2025: 14; 2026: 13.

163 papers.

Sort:

By area:

  1. 2022Unsupervised Semantic Segmentation by Distilling Feature Correspondencescited by 241
  2. 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attentioncited by 233
  3. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 219
  4. 2023Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusioncited by 131
  5. 2022ReCo: Retrieve and Co-segment for Zero-shot Transfercited by 115
  6. 2022Open Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learningcited by 115
  7. 2022Parametric Classification for Generalized Category Discovery: A Baseline Studycited by 113
  8. 2021Large-scale Unsupervised Semantic Segmentationcited by 98
  9. 2023Beyond Appearance: a Semantic Controllable Self-Supervised Learning Framework for Human-Centric Visual Taskscited by 94
  10. 2021Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrastcited by 85
  11. 2022Perceptual Grouping in Contrastive Vision-Language Modelscited by 82
  12. 2022Object discovery and representation networkscited by 80
38 more
  1. 2022NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenescited by 71
  2. 2022Self-Supervised Visual Representation Learning with Semantic Groupingcited by 70
  3. 2024Explore the Potential of CLIP for Training-Free Open Vocabulary Semantic Segmentationcited by 61
  4. 2022Semi-supervised Semantic Segmentation with Error Localization Networkcited by 57
  5. 2022Discovering Object Masks with Transformers for Unsupervised Semantic Segmentationcited by 54
  6. 2022Unsupervised Semantic Segmentation with Self-supervised Object-centric Representationscited by 54
  7. 2025ConceptAttention: Diffusion Transformers Learn Highly Interpretable Featurescited by 48
  8. 2022A Survey on Label-efficient Deep Image Segmentation: Bridging the Gap between Weak Supervision and Dense Predictioncited by 46
  9. 2023CoDet: Co-Occurrence Guided Region-Word Alignment for Open-Vocabulary Object Detectioncited by 46
  10. 2022ACSeg: Adaptive Conceptualization for Unsupervised Semantic Segmentationcited by 42
  11. 2024EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Modelscited by 42
  12. 2023Leveraging Hidden Positives for Unsupervised Semantic Segmentationcited by 40
  13. 2023VDD: Varied Drone Dataset for Semantic Segmentationcited by 39
  14. 2021Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AIcited by 38
  15. 2021TransFGU: A Top-down Approach to Fine-Grained Unsupervised Semantic Segmentationcited by 38
  16. 2022Self-Guided Diffusion Modelscited by 36
  17. 2022Peekaboo: Text to Image Diffusion Models are Zero-Shot Segmentorscited by 34
  18. 2023CLIP-S4: Language-Guided Self-Supervised Semantic Segmentationcited by 34
  19. 2023GrowSP: Unsupervised Semantic Segmentation of 3D Point Cloudscited by 34
  20. 2023Unsupervised Universal Image Segmentationcited by 34
  21. 2021Novel Class Discovery in Semantic Segmentationcited by 31
  22. 2024EAGLE: Eigen Aggregation Learning for Object-Centric Unsupervised Semantic Segmentationcited by 27
  23. 2023Image Clustering Conditioned on Text Criteriacited by 26
  24. 2024DiffCut: Catalyzing Zero-Shot Semantic Segmentation with Diffusion Features and Recursive Normalized Cutcited by 26
  25. 2022OGC: Unsupervised 3D Object Segmentation from Rigid Dynamics of Point Cloudscited by 24
  26. 2023Component-aware anomaly detection framework for adjustable and logical industrial visual inspectioncited by 22
  27. 2023CrOC: Cross-View Online Clustering for Dense Visual Representation Learningcited by 21
  28. 2022NamedMask: Distilling Segmenters from Complementary Foundation Modelscited by 20
  29. 2022Location-Aware Self-Supervised Transformers for Semantic Segmentationcited by 20
  30. 2022Drive&Segment: Unsupervised Semantic Segmentation of Urban Scenes via Cross-modal Distillationcited by 19
  31. 2023Unsupervised Semantic Segmentation Through Depth-Guided Feature Correlation and Samplingcited by 19
  32. 2023TagCLIP: A Local-to-Global Framework to Enhance Open-Vocabulary Multi-Label Classification of CLIP Without Trainingcited by 19
  33. 2025SAMRefiner: Taming Segment Anything Model for Universal Mask Refinementcited by 19
  34. 2023Few-Shot Panoptic Segmentation With Foundation Modelscited by 18
  35. 2023Causal Unsupervised Semantic Segmentationcited by 18
  36. 2023SmooSeg: Smoothness Prior for Unsupervised Semantic Segmentationcited by 18
  37. 2024A Simple Latent Diffusion Approach for Panoptic Segmentation and Mask Inpaintingcited by 18
  38. 2023CrIBo: Self-Supervised Learning via Cross-Image Object-Level Bootstrappingcited by 17

50 most cited of 163 All 163 (JSON)