mlsanityData to 7 Oct 2026

Cross Language Image Matching for Weakly Supervised Semantic Segmentation

arXiv 2203.02668 ↗First posted 5 Mar 2022cs.CV cs.CLcited by 72

Not on the map. Why?

Abstract (arXiv)

It has been widely known that CAM (Class Activation Map) usually only activates discriminative object regions and falsely includes lots of object-related backgrounds. As only a fixed set of image-level object labels are available to the WSSS (weakly supervised semantic segmentation) model, it could be very difficult to suppress those diverse background regions consisting of open set objects. In this paper, we propose a novel Cross Language Image Matching (CLIMS) framework, based on the recently introduced Contrastive Language-Image Pre-training (CLIP) model, for WSSS. The core idea of our framework is to introduce natural language supervision to activate more complete object regions and suppress closely-related open background regions. In particular, we design object, background region and text label matching losses to guide the model to excite more reasonable object regions for CAM of each category. In addition, we design a co-occurring background suppression loss to prevent the model from activating closely-related background regions, with a predefined set of class-related background text descriptions. These designs enable the proposed CLIMS to generate a more complete and compact activation map for the target objects. Extensive experiments on PASCAL VOC2012 dataset show that our CLIMS significantly outperforms the previous state-of-the-art methods.

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). Builds on 34 of its 48 references that are in the map. First posted on arXiv 5 Mar 2022.

Builds on

34 of 48 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  5. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  6. 2014Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFscited by 552
  7. 2020Semi-Supervised Semantic Segmentation with Cross-Consistency Trainingcited by 375
  8. 2016ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentationcited by 330
  9. 2018Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentationcited by 321
  10. 2020Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentationcited by 299
  11. 2015What's the Point: Semantic Segmentation with Point Supervisioncited by 297
  12. 2019Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relationscited by 245
22 more
  1. 2015BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentationcited by 238
  2. 2020Causal Intervention for Weakly-Supervised Semantic Segmentationcited by 229
  3. 2016Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentationcited by 198
  4. 2015Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentationcited by 189
  5. 2016Wider or Deeper: Revisiting the ResNet Model for Visual Recognitioncited by 184
  6. 2018Tell Me Where to Look: Guided Attention Inference Networkcited by 168
  7. 2019FickleNet: Weakly and Semi-supervised Semantic Image Segmentation using Stochastic Inferencecited by 162
  8. 2018Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentationcited by 145
  9. 2020Weakly-Supervised Semantic Segmentation via Sub-category Explorationcited by 145
  10. 2020Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentationcited by 144
  11. 2021Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentationcited by 134
  12. 2021Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentationcited by 127
  13. 2018Learning random-walk label propagation for weakly-supervised semantic segmentationcited by 101
  14. 2015Constrained Convolutional Neural Networks for Weakly Supervised Segmentationcited by 100
  15. 2018Self-Erasing Network for Integral Object Attentioncited by 95
  16. 2021Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentationcited by 95
  17. 2021Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentationcited by 77
  18. 2021Discriminative Region Suppression for Weakly-Supervised Semantic Segmentationcited by 50
  19. 2021Affinity Attention Graph Neural Network for Weakly Supervised Semantic Segmentationcited by 47
  20. 2020Weakly-Supervised Semantic Segmentation by Iterative Affinity Learningcited by 37
  21. 2021Puzzle-CAM: Improved localization via matching partial and full featurescited by 37
  22. 2020Geometry Constrained Weakly Supervised Object Localizationcited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 28; 2024: 18; 2025: 11; 2026: 7.

72 papers.

Sort:

By area:

  1. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  2. 2022CLIP is Also an Efficient Segmenter: A Text-Driven Approach for Weakly Supervised Semantic Segmentationcited by 118
  3. 2023CLIP goes 3D: Leveraging Prompt Tuning for Language Grounded 3D Recognitioncited by 77
  4. 2023Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmentercited by 76
  5. 2023CLIP as RNN: Segment Countless Visual Concepts without Training Endeavorcited by 69
  6. 2023LOCATE: Localize and Transfer Object Parts for Weakly Supervised Affordance Groundingcited by 64
  7. 2023Segment Anything Model (SAM) Enhanced Pseudo Labels for Weakly Supervised Semantic Segmentationcited by 53
  8. 2022A Survey on Label-efficient Deep Image Segmentation: Bridging the Gap between Weak Supervision and Dense Predictioncited by 46
  9. 2022Max Pooling with Vision Transformers reconciles class and shape in weakly supervised semantic segmentationcited by 41
  10. 2022Expansion and Shrinkage of Localization for Weakly-Supervised Semantic Segmentationcited by 37
  11. 2023Referring Image Segmentation Using Text Supervisioncited by 35
  12. 2023Foundation Model Assisted Weakly Supervised Semantic Segmentationcited by 31
38 more
  1. 2023WeakTr: Exploring Plain Vision Transformer for Weakly-supervised Semantic Segmentationcited by 22
  2. 2023MCTformer+: Multi-Class Token Transformer for Weakly Supervised Semantic Segmentationcited by 22
  3. 2025SAMRefiner: Taming Segment Anything Model for Universal Mask Refinementcited by 19
  4. 2023Background Activation Suppression for Weakly Supervised Object Localization and Semantic Segmentationcited by 17
  5. 2024CLIPer: Hierarchically Improving Spatial Representation of CLIP for Open-Vocabulary Semantic Segmentationcited by 17
  6. 2023Multi-Granularity Denoising and Bidirectional Alignment for Weakly Supervised Semantic Segmentationcited by 16
  7. 2023From Text to Mask: Localizing Entities Using the Attention of Text-to-Image Diffusion Modelscited by 16
  8. 2024Hunting Attributes: Context Prototype-Aware Learning for Weakly Supervised Semantic Segmentationcited by 16
  9. 2025Prototype-Based Image Prompting for Weakly Supervised Histopathological Image Segmentationcited by 16
  10. 2023USAGE: A Unified Seed Area Generation Paradigm for Weakly Supervised Semantic Segmentationcited by 14
  11. 2024Separate and Conquer: Decoupling Co-occurrence via Decomposition and Representation for Weakly Supervised Semantic Segmentationcited by 14
  12. 2024Frozen CLIP: A Strong Backbone for Weakly Supervised Semantic Segmentationcited by 14
  13. 2023MARS: Model-agnostic Biased Object Removal without Additional Supervision for Weakly-Supervised Semantic Segmentationcited by 13
  14. 2023Can CLIP Help Sound Source Localization?cited by 13
  15. 2023TCEIP: Text Condition Embedded Regression Network for Dental Implant Position Predictioncited by 11
  16. 2024iSeg: An Iterative Refinement-based Framework for Training-free Segmentationcited by 10
  17. 2023Learning to Identify Critical States for Reinforcement Learning from Videoscited by 9
  18. 2025Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priorscited by 9
  19. 2024Knowledge Transfer with Simulated Inter-Image Erasing for Weakly Supervised Semantic Segmentationcited by 8
  20. 2025Exploring CLIP's Dense Knowledge for Weakly Supervised Semantic Segmentationcited by 8
  21. 2024Tuning-free Universally-Supervised Semantic Segmentationcited by 6
  22. 2024Cognition Transferring and Decoupling for Text-supervised Egocentric Semantic Segmentationcited by 6
  23. 2024Tackling Ambiguity from Perspective of Uncertainty Inference and Affinity Diversification for Weakly Supervised Semantic Segmentationcited by 4
  24. 2024Fine-grained Background Representation for Weakly Supervised Semantic Segmentationcited by 4
  25. 2024DIAL: Dense Image-text ALignment for Weakly Supervised Semantic Segmentationcited by 4
  26. 2025DOEI: Dual Optimization of Embedding Information for Attention-Enhanced Class Activation Mapscited by 4
  27. 2023Prompting classes: Exploring the Power of Prompt Class Learning in Weakly Supervised Semantic Segmentationcited by 3
  28. 2024Semantic Prompt Learning for Weakly-Supervised Semantic Segmentationcited by 3
  29. 2024Weakly Supervised Co-training with Swapping Assignments for Semantic Segmentationcited by 3
  30. 2022SemFormer: Semantic Guided Activation Transformer for Weakly Supervised Semantic Segmentationcited by 2
  31. 2023Exploit CAM by itself: Complementary Learning System for Weakly Supervised Semantic Segmentationcited by 2
  32. 2023Masked Collaborative Contrast for Weakly Supervised Semantic Segmentationcited by 2
  33. 2023Boosting Weakly-Supervised Image Segmentation via Representation, Transform, and Compensatorcited by 2
  34. 2024Boosting Medical Image-based Cancer Detection via Text-guided Supervision from Reportscited by 2
  35. 2024Pixel-Level Domain Adaptation: A New Perspective for Enhancing Weakly Supervised Semantic Segmentationcited by 2
  36. 2025TeD-Loc: Text Distillation for Weakly Supervised Object Localizationcited by 2
  37. 2025Multimodal Prototype Alignment for Semi-supervised Pathology Image Segmentationcited by 2
  38. 2025NERVE: Neighbourhood & Entropy-guided Random-walk for training free open-Vocabulary sEgmentationcited by 2

50 most cited of 72 All 72 (JSON)