mlsanityData to 14 Oct 2026

Spatial Structure Constraints for Weakly Supervised Semantic Segmentation

arXiv 2401.11122 ↗First posted 20 Jan 2024cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

The image-level label has prevailed in weakly supervised semantic segmentation tasks due to its easy availability. Since image-level labels can only indicate the existence or absence of specific categories of objects, visualization-based techniques have been widely adopted to provide object location clues. Considering class activation maps (CAMs) can only locate the most discriminative part of objects, recent approaches usually adopt an expansion strategy to enlarge the activation area for more integral object localization. However, without proper constraints, the expanded activation will easily intrude into the background region. In this paper, we propose spatial structure constraints (SSC) for weakly supervised semantic segmentation to alleviate the unwanted object over-activation of attention expansion. Specifically, we propose a CAM-driven reconstruction module to directly reconstruct the input image from deep CAM features, which constrains the diffusion of last-layer object attention by preserving the coarse spatial structure of the image content. Moreover, we propose an activation self-modulation module to refine CAMs with finer spatial structure details by enhancing regional consistency. Without external saliency models to provide background clues, our approach achieves 72.7% and 47.0% mIoU on the PASCAL VOC 2012 and COCO datasets, respectively, demonstrating the superiority of our proposed approach.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 71 of its 100 references that are in the map. First posted on arXiv 20 Jan 2024.

Builds on

71 of 100 references.

References · 71

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  8. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  9. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  10. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  11. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  12. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
38 more
  1. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  2. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  3. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 657
  4. 2016ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentationcited by 331
  5. 2018Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentationcited by 321
  6. 2020Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentationcited by 300
  7. 2015What's the Point: Semantic Segmentation with Point Supervisioncited by 298
  8. 2019Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relationscited by 247
  9. 2021Exploring Cross-Image Pixel Contrast for Semantic Segmentationcited by 242
  10. 2015BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentationcited by 238
  11. 2020Causal Intervention for Weakly-Supervised Semantic Segmentationcited by 229
  12. 2016Simple Does It: Weakly Supervised Instance and Semantic Segmentationcited by 227
  13. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 207
  14. 2017Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approachcited by 201
  15. 2016Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentationcited by 198
  16. 2018Adversarial Complementary Learning for Weakly Supervised Object Localizationcited by 166
  17. 2018Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentationcited by 145
  18. 2022Rethinking Semantic Segmentation: A Prototype Viewcited by 140
  19. 2022Multi-class Token Transformer for Weakly Supervised Semantic Segmentationcited by 138
  20. 2021Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentationcited by 134
  21. 2021Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentationcited by 127
  22. 2022Learning Affinity from Attention: End-to-End Weakly-Supervised Semantic Segmentation with Transformerscited by 118
  23. 2022CLIP is Also an Efficient Segmenter: A Text-Driven Approach for Weakly Supervised Semantic Segmentationcited by 118
  24. 2018Learning random-walk label propagation for weakly-supervised semantic segmentationcited by 101
  25. 2019Box-driven Class-wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentationcited by 98
  26. 2021Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentationcited by 95
  27. 2022Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentationcited by 87
  28. 2021Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrastcited by 85
  29. 2021Jo-SRC: A Contrastive Approach for Combating Noisy Labelscited by 80
  30. 2021Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentationcited by 77
  31. 2022Class Re-Activation Maps for Weakly-Supervised Semantic Segmentationcited by 75
  32. 2021Context Decoupling Augmentation for Weakly Supervised Semantic Segmentationcited by 69
  33. 2021Reducing Information Bottleneck for Weakly Supervised Semantic Segmentationcited by 65
  34. 2023Token Contrast for Weakly-Supervised Semantic Segmentationcited by 65
  35. 2022Regional Semantic Contrast and Aggregation for Weakly Supervised Semantic Segmentationcited by 64
  36. 2022Weakly Supervised Semantic Segmentation using Out-of-Distribution Datacited by 58
  37. 2022L2G: A Simple Local-to-Global Knowledge Transfer Framework for Weakly Supervised Semantic Segmentationcited by 57
  38. 2020Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentationcited by 52

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 6; 2026: 5.

17 papers.

Sort:

By area:

  1. 2024Self-Calibrated CLIP for Training-Free Open-Vocabulary Segmentationcited by 34
  2. 2024Knowledge Transfer with Simulated Inter-Image Erasing for Weakly Supervised Semantic Segmentationcited by 8
  3. 2025Semi-supervised Semantic Segmentation with Multi-Constraint Consistency Learningcited by 8
  4. 2025SpectralX: Parameter-efficient Domain Generalization for Spectral Remote Sensing Foundation Modelscited by 5
  5. 2026PCA-Seg: Revisiting Cost Aggregation for Open-Vocabulary Semantic and Part Segmentationcited by 5
  6. 2025Efficient Token Compression for Vision Transformer with Spatial Information Preservedcited by 4
  7. 2024Relating CNN-Transformer Fusion Network for Change Detectioncited by 3
  8. 2024Pixel-Level Domain Adaptation: A New Perspective for Enhancing Weakly Supervised Semantic Segmentationcited by 2
  9. 2025Background Matters: A Cross-view Bidirectional Modeling Framework for Semi-supervised Medical Image Segmentationcited by 2
  10. 20253D Weakly Supervised Semantic Segmentation via Class-Aware and Geometry-Guided Pseudo-Label Refinementcited by 1
  11. 2024Anti-Collapse Loss for Deep Metric Learning Based on Coding Rate Metriccited by 0
  12. 2024Text-Driven Weakly Supervised OCT Lesion Segmentation with Structural Guidancecited by 0
5 more
  1. 2025Region-CAM: Towards Accurate Object Regions in Class Activation Maps for Weakly Supervised Learning Taskscited by 0
  2. 2026DiCLIP: Diffusion Model Enhances CLIP's Dense Knowledge for Weakly Supervised Semantic Segmentationcited by 0
  3. 2026Widest-Path Reachability Fields for Connectivity-Preserving Slender Structure Segmentationcited by 0
  4. 2026Contextrast++: Robust Multi-Scale Contextual Contrastive Learning for Semantic Segmentationcited by 0
  5. 2026SomaNet: Weakly Supervised Learning for Instance Soma Segmentation in 3D Electron Microscopy with Partial Annotationscited by 0