# Extracting Class Activation Maps from Non-Discriminative Features as well

arXiv: https://arxiv.org/abs/2303.10334 · HTML: https://mlsanity.com/paper/2303.10334/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 53 references that are in the map.
- First posted on arXiv 18 Mar 2023.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1803.10464/) (2018)
- [Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2004.04581/) (2020)
- [Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations](https://mlsanity.com/paper/1904.05044/) (2019)
- [Causal Intervention for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2009.12547/) (2020)
- [A Simple Pooling-Based Design for Real-Time Salient Object Detection](https://mlsanity.com/paper/1904.09569/) (2019)
- [Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach](https://mlsanity.com/paper/1703.08448/) (2017)
- [Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation](https://mlsanity.com/paper/1603.06098/) (2016)
- [Deeply supervised salient object detection with short connections](https://mlsanity.com/paper/1611.04849/) (2016)
- [Wider or Deeper: Revisiting the ResNet Model for Visual Recognition](https://mlsanity.com/paper/1611.10080/) (2016)
- [Adversarial Complementary Learning for Weakly Supervised Object Localization](https://mlsanity.com/paper/1804.06962/) (2018)
- [FickleNet: Weakly and Semi-supervised Semantic Image Segmentation using Stochastic Inference](https://mlsanity.com/paper/1902.10421/) (2019)
- [Weakly-Supervised Semantic Segmentation via Sub-category Exploration](https://mlsanity.com/paper/2008.01183/) (2020)
- [Multi-class Token Transformer for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.02891/) (2022)
- [Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2103.08896/) (2021)
- [Pyramid Feature Attention Network for Saliency detection](https://mlsanity.com/paper/1903.00179/) (2019)
- [Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2105.08965/) (2021)
- [Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.02909/) (2022)
- [Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast](https://mlsanity.com/paper/2110.07110/) (2021)
- [Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2107.11787/) (2021)
- [Class Re-Activation Maps for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.00962/) (2022)
- [Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2110.06530/) (2021)
- [Regional Semantic Contrast and Aggregation for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.09653/) (2022)

## Built on by (most cited first, 30 of 30)

- [WeakTr: Exploring Plain Vision Transformer for Weakly-supervised Semantic Segmentation](https://mlsanity.com/paper/2304.01184/) (2023)
- [MCTformer+: Multi-Class Token Transformer for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2308.03005/) (2023)
- [Weakly-Supervised Semantic Segmentation with Image-Level Labels: from Traditional Models to Foundation Models](https://mlsanity.com/paper/2310.13026/) (2023)
- [Spatial Structure Constraints for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2401.11122/) (2024)
- [Hunting Attributes: Context Prototype-Aware Learning for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2403.07630/) (2024)
- [MoRe: Class Patch Attention Needs Regularization for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2412.11076/) (2024)
- [Knowledge Transfer with Simulated Inter-Image Erasing for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2407.02768/) (2024)
- [Exploring CLIP's Dense Knowledge for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2503.20826/) (2025)
- [Question-Answer Cross Language Image Matching for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2401.09883/) (2024)
- [CAPE: CAM as a Probabilistic Ensemble for Enhanced DNN Interpretation](https://mlsanity.com/paper/2404.02388/) (2024)
- [Efficient and Accurate Explanation Estimation with Distribution Compression](https://mlsanity.com/paper/2406.18334/) (2024)
- [DIAL: Dense Image-text ALignment for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2409.15801/) (2024)
- [Exact: Exploring Space-Time Perceptive Clues for Weakly Supervised Satellite Image Time Series Semantic Segmentation](https://mlsanity.com/paper/2412.03968/) (2024)
- [DOEI: Dual Optimization of Embedding Information for Attention-Enhanced Class Activation Maps](https://mlsanity.com/paper/2502.15885/) (2025)
- [Calibrating Undisciplined Over-Smoothing in Transformer for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2305.03112/) (2023)
- [Semantic Prompt Learning for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2401.11791/) (2024)
- [Weakly Supervised Co-training with Swapping Assignments for Semantic Segmentation](https://mlsanity.com/paper/2402.17891/) (2024)
- [Masked Collaborative Contrast for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2305.08491/) (2023)
- [Weakly Supervised Semantic Segmentation by Knowledge Graph Inference](https://mlsanity.com/paper/2309.14057/) (2023)
- [Mitigating Spurious Correlations in Weakly Supervised Semantic Segmentation via Cross-architecture Consistency Regularization](https://mlsanity.com/paper/2507.21959/) (2025)
- [Localized Region Guidance for Class Activation Mapping in WSSS](https://mlsanity.com/paper/2509.12496/) (2025)
- [DynaLay: An Introspective Approach to Dynamic Layer Selection for Deep Networks](https://mlsanity.com/paper/2312.12781/) (2023)
- [Background Noise Reduction of Attention Map for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2404.03394/) (2024)
- [Emerging Trends in Pseudo-Label Refinement for Weakly Supervised Semantic Segmentation with Image-Level Supervision](https://mlsanity.com/paper/2507.21587/) (2025)
- [TinyDrop: Tiny Model Guided Token Dropping for Vision Transformers](https://mlsanity.com/paper/2509.03379/) (2025)
- [Instance Relation Learning Network with Label Knowledge Propagation for Few-shot Multi-label Intent Detection](https://mlsanity.com/paper/2510.07776/) (2025)
- [ModuSeg: Decoupling Object Discovery and Semantic Retrieval for Training-Free Weakly Supervised Segmentation](https://mlsanity.com/paper/2604.07021/) (2026)
- [DiCLIP: Diffusion Model Enhances CLIP's Dense Knowledge for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2605.04593/) (2026)
- [APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigms](https://mlsanity.com/paper/2606.30577/) (2026)
- [Class Activation Mapping in Explainable Computer Vision: A Method-Centered Review of CNN, Transformer, and Foundation-Model-Era Visual Explanations](https://mlsanity.com/paper/2608.12299/) (2026)

## Abstract (arXiv)

Extracting class activation maps (CAM) from a classification model often results in poor coverage on foreground objects, i.e., only the discriminative region (e.g., the "head" of "sheep") is recognized and the rest (e.g., the "leg" of "sheep") mistakenly as background. The crux behind is that the weight of the classifier (used to compute CAM) captures only the discriminative features of objects. We tackle this by introducing a new computation method for CAM that explicitly captures non-discriminative features as well, thereby expanding CAM to cover whole objects. Specifically, we omit the last pooling layer of the classification model, and perform clustering on all local features of an object class, where "local" means "at a spatial pixel position". We call the resultant K cluster centers local prototypes - represent local semantics like the "head", "leg", and "body" of "sheep". Given a new image of the class, we compare its unpooled features to every prototype, derive K similarity matrices, and then aggregate them into a heatmap (i.e., our CAM). Our CAM thus captures all local features of the class without discrimination. We evaluate it in the challenging tasks of weakly-supervised semantic segmentation (WSSS), and plug it in multiple state-of-the-art WSSS methods, such as MCTformer and AMN, by simply replacing their original CAM with ours. Our extensive experiments on standard WSSS benchmarks (PASCAL VOC and MS COCO) show the superiority of our method: consistent improvements with little computational overhead.