# Regional Semantic Contrast and Aggregation for Weakly Supervised Semantic Segmentation

arXiv: https://arxiv.org/abs/2203.09653 · HTML: https://mlsanity.com/paper/2203.09653/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- Builds on 71 of its 87 references that are in the map.
- First posted on arXiv 17 Mar 2022.

## Builds on (most cited first, 30 of 71)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [Dense Contrastive Learning for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2011.09157/) (2020)
- [ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1604.05144/) (2016)
- [Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1803.10464/) (2018)
- [Contrastive learning of global and local features for medical image segmentation with limited annotations](https://mlsanity.com/paper/2006.10511/) (2020)
- [Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2004.04581/) (2020)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/2011.10043/) (2020)
- [Context Encoding for Semantic Segmentation](https://mlsanity.com/paper/1803.08904/) (2018)
- [Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations](https://mlsanity.com/paper/1904.05044/) (2019)
- [Exploring Cross-Image Pixel Contrast for Semantic Segmentation](https://mlsanity.com/paper/2101.11939/) (2021)
- [BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1503.01640/) (2015)
- [Causal Intervention for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2009.12547/) (2020)

## Built on by (most cited first, 30 of 64)

- [CLIP is Also an Efficient Segmenter: A Text-Driven Approach for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2212.09506/) (2022)
- [Token Contrast for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2303.01267/) (2023)
- [LOCATE: Localize and Transfer Object Parts for Weakly Supervised Affordance Grounding](https://mlsanity.com/paper/2303.09665/) (2023)
- [Segment Anything Model (SAM) Enhanced Pseudo Labels for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2305.05803/) (2023)
- [Tree Energy Loss: Towards Sparsely Annotated Semantic Segmentation](https://mlsanity.com/paper/2203.10739/) (2022)
- [A Survey on Label-efficient Deep Image Segmentation: Bridging the Gap between Weak Supervision and Dense Prediction](https://mlsanity.com/paper/2207.01223/) (2022)
- [Max Pooling with Vision Transformers reconciles class and shape in weakly supervised semantic segmentation](https://mlsanity.com/paper/2210.17400/) (2022)
- [Hierarchical Semantic Contrast for Scene-aware Video Anomaly Detection](https://mlsanity.com/paper/2303.13051/) (2023)
- [Foundation Model Assisted Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2312.03585/) (2023)
- [Extracting Class Activation Maps from Non-Discriminative Features as well](https://mlsanity.com/paper/2303.10334/) (2023)
- [Out-of-Candidate Rectification for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2211.12268/) (2022)
- [MCTformer+: Multi-Class Token Transformer for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2308.03005/) (2023)
- [Weakly-Supervised Semantic Segmentation with Image-Level Labels: from Traditional Models to Foundation Models](https://mlsanity.com/paper/2310.13026/) (2023)
- [Spatial Structure Constraints for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2401.11122/) (2024)
- [Hunting Attributes: Context Prototype-Aware Learning for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2403.07630/) (2024)
- [TransVCL: Attention-enhanced Video Copy Localization Network with Flexible Supervision](https://mlsanity.com/paper/2211.13090/) (2022)
- [Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models](https://mlsanity.com/paper/2311.17095/) (2023)
- [Frozen CLIP: A Strong Backbone for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2406.11189/) (2024)
- [Separate and Conquer: Decoupling Co-occurrence via Decomposition and Representation for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2402.18467/) (2024)
- [MARS: Model-agnostic Biased Object Removal without Additional Supervision for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2304.09913/) (2023)
- [Semi-Supervised Semantic Segmentation with Cross Teacher Training](https://mlsanity.com/paper/2209.01327/) (2022)
- [SIM: Semantic-aware Instance Mask Generation for Box-Supervised Instance Segmentation](https://mlsanity.com/paper/2303.08578/) (2023)
- [CPM: Class-conditional Prompting Machine for Audio-visual Segmentation](https://mlsanity.com/paper/2407.05358/) (2024)
- [Timestamp-supervised Wearable-based Activity Segmentation and Recognition with Contrastive Learning and Order-Preserving Optimal Transport](https://mlsanity.com/paper/2310.09114/) (2023)
- [Learning without Exact Guidance: Updating Large-scale High-resolution Land Cover Maps from Low-resolution Historical Labels](https://mlsanity.com/paper/2403.02746/) (2024)
- [Knowledge Transfer with Simulated Inter-Image Erasing for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2407.02768/) (2024)
- [WeakMedSAM: Weakly-Supervised Medical Image Segmentation via SAM with Sub-Class Exploration and Prompt Affinity Mining](https://mlsanity.com/paper/2503.04106/) (2025)
- [Exploring CLIP's Dense Knowledge for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2503.20826/) (2025)
- [Centered Self-Attention Layers](https://mlsanity.com/paper/2306.01610/) (2023)
- [MS-Former: Memory-Supported Transformer for Weakly Supervised Change Detection with Patch-Level Annotations](https://mlsanity.com/paper/2311.09726/) (2023)

## Abstract (arXiv)

Learning semantic segmentation from weakly-labeled (e.g., image tags only) data is challenging since it is hard to infer dense object regions from sparse semantic tags. Despite being broadly studied, most current efforts directly learn from limited semantic annotations carried by individual image or image pairs, and struggle to obtain integral localization maps. Our work alleviates this from a novel perspective, by exploring rich semantic contexts synergistically among abundant weakly-labeled training data for network learning and inference. In particular, we propose regional semantic contrast and aggregation (RCA) . RCA is equipped with a regional memory bank to store massive, diverse object patterns appearing in training data, which acts as strong support for exploration of dataset-level semantic structure. Particularly, we propose i) semantic contrast to drive network learning by contrasting massive categorical object regions, leading to a more holistic object pattern understanding, and ii) semantic aggregation to gather diverse relational contexts in the memory to enrich semantic representations. In this manner, RCA earns a strong capability of fine-grained semantic understanding, and eventually establishes new state-of-the-art results on two popular benchmarks, i.e., PASCAL VOC 2012 and COCO 2014.