mlsanityData to 7 Oct 2026

Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast

arXiv 2110.07110 ↗First posted 14 Oct 2021cs.CVcited by 85

Not on the map. Why?

Abstract (arXiv)

Though image-level weakly supervised semantic segmentation (WSSS) has achieved great progress with Class Activation Maps (CAMs) as the cornerstone, the large supervision gap between classification and segmentation still hampers the model to generate more complete and precise pseudo masks for segmentation. In this study, we propose weakly-supervised pixel-to-prototype contrast that can provide pixel-level supervisory signals to narrow the gap. Guided by two intuitive priors, our method is executed across different views and within per single view of an image, aiming to impose cross-view feature semantic consistency regularization and facilitate intra(inter)-class compactness(dispersion) of the feature space. Our method can be seamlessly incorporated into existing WSSS models without any changes to the base networks and does not incur any extra inference burden. Extensive experiments manifest that our method consistently improves two strong baselines by large margins, demonstrating the effectiveness. Specifically, built on top of SEAM, we improve the initial seed mIoU on PASCAL VOC 2012 from 55.4% to 61.5%. Moreover, armed with our method, we increase the segmentation mIoU of EPS from 70.8% to 73.6%, achieving new state-of-the-art.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Builds on 66 of its 81 references that are in the map. First posted on arXiv 14 Oct 2021.

Builds on

66 of 81 references.

References · 66

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  5. 2020Supervised Contrastive Learningcited by 2,803
  6. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  7. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  8. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  9. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  10. 2020Self-supervised Learning: Generative or Contrastivecited by 693
  11. 2020Image Segmentation Using Deep Learning: A Surveycited by 659
  12. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 636
38 more
  1. 2020Contrastive Learning with Hard Negative Samplescited by 619
  2. 2020A Survey on Contrastive Self-supervised Learningcited by 553
  3. 2014Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFscited by 552
  4. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  5. 2021Semi-Supervised Semantic Segmentation with Cross Pseudo Supervisioncited by 462
  6. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  7. 2020Hard Negative Mixing for Contrastive Learningcited by 399
  8. 2020Debiased Contrastive Learningcited by 354
  9. 2016ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentationcited by 330
  10. 2018Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentationcited by 321
  11. 2020Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentationcited by 299
  12. 2015What's the Point: Semantic Segmentation with Point Supervisioncited by 297
  13. 2019Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relationscited by 245
  14. 2020Causal Intervention for Weakly-Supervised Semantic Segmentationcited by 229
  15. 2017Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approachcited by 201
  16. 2016Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentationcited by 198
  17. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  18. 2016Wider or Deeper: Revisiting the ResNet Model for Visual Recognitioncited by 184
  19. 2018Tell Me Where to Look: Guided Attention Inference Networkcited by 168
  20. 2018Adversarial Complementary Learning for Weakly Supervised Object Localizationcited by 166
  21. 2021PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clusteringcited by 163
  22. 2019FickleNet: Weakly and Semi-supervised Semantic Image Segmentation using Stochastic Inferencecited by 162
  23. 2018Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentationcited by 145
  24. 2020Weakly-Supervised Semantic Segmentation via Sub-category Explorationcited by 145
  25. 2020Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentationcited by 144
  26. 2021Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentationcited by 134
  27. 2021Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentationcited by 127
  28. 2021Semi-Supervised Semantic Segmentation with Pixel-Level Contrastive Learning from a Class-wise Memory Bankcited by 125
  29. 2018On Regularized Losses for Weakly-supervised CNN Segmentationcited by 121
  30. 2020Guided Collaborative Training for Pixel-wise Semi-Supervised Learningcited by 120
  31. 2018Self-Erasing Network for Integral Object Attentioncited by 95
  32. 2021Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentationcited by 95
  33. 2020Contrastive Learning for Label-Efficient Semantic Segmentationcited by 93
  34. 2019Reliability Does Matter: An End-to-End Weakly Supervised Semantic Segmentation Approachcited by 88
  35. 2018Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Featurescited by 81
  36. 2018CIAN: Cross-Image Affinity Net for Weakly Supervised Semantic Segmentationcited by 80
  37. 2015STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentationcited by 78
  38. 2021Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentationcited by 77

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 29; 2024: 27; 2025: 19; 2026: 3.

85 papers.

Sort:

By area:

  1. 2022CLIP is Also an Efficient Segmenter: A Text-Driven Approach for Weakly Supervised Semantic Segmentationcited by 118
  2. 2023Token Contrast for Weakly-Supervised Semantic Segmentationcited by 64
  3. 2023Segment Anything Model (SAM) Enhanced Pseudo Labels for Weakly Supervised Semantic Segmentationcited by 53
  4. 2022Max Pooling with Vision Transformers reconciles class and shape in weakly supervised semantic segmentationcited by 41
  5. 2023Hierarchical Semantic Contrast for Scene-aware Video Anomaly Detectioncited by 39
  6. 2023Foundation Model Assisted Weakly Supervised Semantic Segmentationcited by 31
  7. 2023Extracting Class Activation Maps from Non-Discriminative Features as wellcited by 30
  8. 2022Out-of-Candidate Rectification for Weakly Supervised Semantic Segmentationcited by 26
  9. 2023Self Correspondence Distillation for End-to-End Weakly-Supervised Semantic Segmentationcited by 26
  10. 2023MCTformer+: Multi-Class Token Transformer for Weakly Supervised Semantic Segmentationcited by 22
  11. 2023An Alternative to WSSS? An Empirical Study of the Segment Anything Model (SAM) on Weakly-Supervised Semantic Segmentation Problemscited by 19
  12. 2023Image Augmentation with Controlled Diffusion for Weakly-Supervised Semantic Segmentationcited by 19
38 more
  1. 2023Weakly-Supervised Semantic Segmentation with Image-Level Labels: from Traditional Models to Foundation Modelscited by 19
  2. 2024SFC: Shared Feature Calibration in Weakly Supervised Semantic Segmentationcited by 19
  3. 2021Weakly Supervised Semantic Segmentation via Alternative Self-Dual Teachingcited by 18
  4. 2023Background Activation Suppression for Weakly Supervised Object Localization and Semantic Segmentationcited by 17
  5. 2024Spatial Structure Constraints for Weakly Supervised Semantic Segmentationcited by 17
  6. 2024DuPL: Dual Student with Trustworthy Progressive Learning for Robust Weakly Supervised Semantic Segmentationcited by 17
  7. 2024Hunting Attributes: Context Prototype-Aware Learning for Weakly Supervised Semantic Segmentationcited by 16
  8. 2024Prompt Categories Cluster for Weakly Supervised Semantic Segmentationcited by 16
  9. 2023Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Modelscited by 15
  10. 2024Adaptive Patch Contrast for Weakly Supervised Semantic Segmentationcited by 15
  11. 2023USAGE: A Unified Seed Area Generation Paradigm for Weakly Supervised Semantic Segmentationcited by 14
  12. 2024Separate and Conquer: Decoupling Co-occurrence via Decomposition and Representation for Weakly Supervised Semantic Segmentationcited by 14
  13. 2024Frozen CLIP: A Strong Backbone for Weakly Supervised Semantic Segmentationcited by 14
  14. 2023MARS: Model-agnostic Biased Object Removal without Additional Supervision for Weakly-Supervised Semantic Segmentationcited by 13
  15. 2023Contrast, Stylize and Adapt: Unsupervised Contrastive Learning Framework for Domain Adaptive Semantic Segmentationcited by 11
  16. 2023Timestamp-supervised Wearable-based Activity Segmentation and Recognition with Contrastive Learning and Order-Preserving Optimal Transportcited by 10
  17. 2023Top-K Pooling with Patch Contrastive Learning for Weakly-Supervised Semantic Segmentationcited by 10
  18. 2025GeoPix: Multi-Modal Large Language Model for Pixel-level Image Understanding in Remote Sensingcited by 10
  19. 2024MoRe: Class Patch Attention Needs Regularization for Weakly Supervised Semantic Segmentationcited by 9
  20. 2023Using Unreliable Pseudo-Labels for Label-Efficient Semantic Segmentationcited by 8
  21. 2024Knowledge Transfer with Simulated Inter-Image Erasing for Weakly Supervised Semantic Segmentationcited by 8
  22. 2024ReCLIP++: Learn to Rectify the Bias of CLIP for Unsupervised Semantic Segmentationcited by 8
  23. 2025WeakMedSAM: Weakly-Supervised Medical Image Segmentation via SAM with Sub-Class Exploration and Prompt Affinity Miningcited by 8
  24. 2025Exploring CLIP's Dense Knowledge for Weakly Supervised Semantic Segmentationcited by 8
  25. 2023Pulling Target to Source: A New Perspective on Domain Adaptive Semantic Segmentationcited by 7
  26. 2023Centered Self-Attention Layerscited by 7
  27. 2023Prototypical Kernel Learning and Open-set Foreground Perception for Generalized Few-shot Semantic Segmentationcited by 6
  28. 2025No Thing, Nothing: Highlighting Safety-Critical Classes for Robust LiDAR Semantic Segmentation in Adverse Weathercited by 5
  29. 2025Uncertainty-aware Cross-training for Semi-supervised Medical Image Segmentationcited by 5
  30. 2022Information-guided pixel augmentation for pixel-wise contrastive learningcited by 4
  31. 2024Fine-grained Background Representation for Weakly Supervised Semantic Segmentationcited by 4
  32. 2024WPS-SAM: Towards Weakly-Supervised Part Segmentation with Foundation Modelscited by 4
  33. 2024Subspace Prototype Guidance for Mitigating Class Imbalance in Point Cloud Semantic Segmentationcited by 4
  34. 2024DIAL: Dense Image-text ALignment for Weakly Supervised Semantic Segmentationcited by 4
  35. 2024Exact: Exploring Space-Time Perceptive Clues for Weakly Supervised Satellite Image Time Series Semantic Segmentationcited by 4
  36. 2025Dual Cross-image Semantic Consistency with Self-aware Pseudo Labeling for Semi-supervised Medical Image Segmentationcited by 4
  37. 2022Importance Sampling CAMs for Weakly-Supervised Segmentationcited by 3
  38. 2023Calibrating Undisciplined Over-Smoothing in Transformer for Weakly Supervised Semantic Segmentationcited by 3

50 most cited of 85 All 85 (JSON)