# Puzzle-CAM: Improved localization via matching partial and full features

arXiv: https://arxiv.org/abs/2101.11253 · HTML: https://mlsanity.com/paper/2101.11253/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 18 references that are in the map.
- First posted on arXiv 27 Jan 2021.

## Builds on (most cited first, 13 of 13)

- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](https://mlsanity.com/paper/1412.7062/) (2014)
- [ResNeSt: Split-Attention Networks](https://mlsanity.com/paper/2004.08955/) (2020)
- [ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1604.05144/) (2016)
- [Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1803.10464/) (2018)
- [Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2004.04581/) (2020)
- [Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations](https://mlsanity.com/paper/1904.05044/) (2019)
- [Simple Does It: Weakly Supervised Instance and Semantic Segmentation](https://mlsanity.com/paper/1603.07485/) (2016)
- [FickleNet: Weakly and Semi-supervised Semantic Image Segmentation using Stochastic Inference](https://mlsanity.com/paper/1902.10421/) (2019)
- [Self-Erasing Network for Integral Object Attention](https://mlsanity.com/paper/1810.09821/) (2018)

## Built on by (most cited first, 30 of 37)

- [GroupViT: Semantic Segmentation Emerges from Text Supervision](https://mlsanity.com/paper/2202.11094/) (2022)
- [Scaling Open-Vocabulary Image Segmentation with Image-Level Labels](https://mlsanity.com/paper/2112.12143/) (2021)
- [Cross Language Image Matching for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.02668/) (2022)
- [Segment Anything Model (SAM) Enhanced Pseudo Labels for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2305.05803/) (2023)
- [ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes](https://mlsanity.com/paper/2106.02740/) (2021)
- [TransCAM: Transformer Attention-based CAM Refinement for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.07239/) (2022)
- [Weakly-Supervised Semantic Segmentation with Visual Words Learning and Hybrid Pooling](https://mlsanity.com/paper/2202.04812/) (2022)
- [Background Activation Suppression for Weakly Supervised Object Localization and Semantic Segmentation](https://mlsanity.com/paper/2309.12943/) (2023)
- [MARS: Model-agnostic Biased Object Removal without Additional Supervision for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2304.09913/) (2023)
- [Mixed Supervised Object Detection by Transferring Mask Prior and Semantic Similarity](https://mlsanity.com/paper/2110.14191/) (2021)
- [Contrastive learning of Class-agnostic Activation Map for Weakly Supervised Object Localization and Semantic Segmentation](https://mlsanity.com/paper/2203.13505/) (2022)
- [Mixed-UNet: Refined Class Activation Mapping for Weakly-Supervised Semantic Segmentation with Multi-scale Inference](https://mlsanity.com/paper/2205.04227/) (2022)
- [Cross-Model Consensus of Explanations and Beyond for Image Classification Models: An Empirical Study](https://mlsanity.com/paper/2109.00707/) (2021)
- [SESS: Saliency Enhancing with Scaling and Sliding](https://mlsanity.com/paper/2207.01769/) (2022)
- [Dual Progressive Transformations for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2209.15211/) (2022)
- [Exploring Limits of Diffusion-Synthetic Training with Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2309.01369/) (2023)
- [DHR: Dual Features-Driven Hierarchical Rebalancing in Inter- and Intra-Class Regions for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2404.00380/) (2024)
- [ISIM: Iterative Self-Improved Model for Weakly Supervised Segmentation](https://mlsanity.com/paper/2211.12455/) (2022)
- [One-shot Weakly-Supervised Segmentation in Medical Images](https://mlsanity.com/paper/2111.10773/) (2021)
- [RecurSeed and EdgePredictMix: Pseudo-Label Refinement Learning for Weakly Supervised Semantic Segmentation across Single- and Multi-Stage Frameworks](https://mlsanity.com/paper/2204.06754/) (2022)
- [Object-Aware Self-supervised Multi-Label Learning](https://mlsanity.com/paper/2205.07028/) (2022)
- [Single-cell Subcellular Protein Localisation Using Novel Ensembles of Diverse Deep Architectures](https://mlsanity.com/paper/2205.09841/) (2022)
- [ExpNet: A unified network for Expert-Level Classification](https://mlsanity.com/paper/2211.15672/) (2022)
- [ReFit: A Framework for Refinement of Weakly Supervised Semantic Segmentation using Object Border Fitting for Medical Images](https://mlsanity.com/paper/2303.07853/) (2023)
- [BroadCAM: Outcome-agnostic Class Activation Mapping for Small-scale Weakly Supervised Applications](https://mlsanity.com/paper/2309.03509/) (2023)
- [Temporal-consistent CAMs for Weakly Supervised Video Segmentation in Waste Sorting](https://mlsanity.com/paper/2502.01455/) (2025)
- [Towards Better Guided Attention and Human Knowledge Insertion in Deep Convolutional Neural Networks](https://mlsanity.com/paper/2210.11177/) (2022)
- [SILOP: An Automated Framework for Semantic Segmentation Using Image Labels Based on Object Perimeters](https://mlsanity.com/paper/2303.07892/) (2023)
- [ISLE: A Framework for Image Level Semantic Segmentation Ensemble](https://mlsanity.com/paper/2303.07898/) (2023)
- [P-NOC: adversarial training of CAM generating networks for robust weakly supervised semantic segmentation priors](https://mlsanity.com/paper/2305.12522/) (2023)

## Abstract (arXiv)

Weakly-supervised semantic segmentation (WSSS) is introduced to narrow the gap for semantic segmentation performance from pixel-level supervision to image-level supervision. Most advanced approaches are based on class activation maps (CAMs) to generate pseudo-labels to train the segmentation network. The main limitation of WSSS is that the process of generating pseudo-labels from CAMs that use an image classifier is mainly focused on the most discriminative parts of the objects. To address this issue, we propose Puzzle-CAM, a process that minimizes differences between the features from separate patches and the whole image. Our method consists of a puzzle module and two regularization terms to discover the most integrated region in an object. Puzzle-CAM can activate the overall region of an object using image-level supervision without requiring extra parameters. % In experiments, Puzzle-CAM outperformed previous state-of-the-art methods using the same labels for supervision on the PASCAL VOC 2012 test dataset. In experiments, Puzzle-CAM outperformed previous state-of-the-art methods using the same labels for supervision on the PASCAL VOC 2012 dataset. Code associated with our experiments is available at https://github.com/OFRIN/PuzzleCAM.