mlsanityData to 14 Oct 2026

Keep CALM and Improve Visual Feature Attribution

arXiv 2106.07861 ↗First posted 15 Jun 2021cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

The class activation mapping, or CAM, has been the cornerstone of feature attribution methods for multiple vision tasks. Its simplicity and effectiveness have led to wide applications in the explanation of visual predictions and weakly-supervised localization tasks. However, CAM has its own shortcomings. The computation of attribution maps relies on ad-hoc calibration steps that are not part of the training computational graph, making it difficult for us to understand the real meaning of the attribution values. In this paper, we improve CAM by explicitly incorporating a latent variable encoding the location of the cue for recognition in the formulation, thereby subsuming the attribution map into the training computational graph. The resulting model, class activation latent mapping, or CALM, is trained with the expectation-maximization algorithm. Our experiments show that CALM identifies discriminative attributes for image classifiers more accurately than CAM and other visual attribution baselines. CALM also shows performance improvements over prior arts on the weakly-supervised object localization benchmarks. Our code is available at https://github.com/naver-ai/calm.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 50 of its 71 references that are in the map. First posted on arXiv 15 Jun 2021.

Builds on

50 of 71 references.

References · 50

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  3. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  4. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  6. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  7. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  8. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  9. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  10. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  11. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  12. 2018Sanity Checks for Saliency Mapscited by 1,156
38 more
  1. 2017SmoothGrad: removing noise by adding noisecited by 1,109
  2. 2020Concept Bottleneck Modelscited by 1,032
  3. 2014Striving for Simplicity: The All Convolutional Netcited by 990
  4. 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
  5. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 939
  6. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
  7. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
  8. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 612
  9. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 582
  10. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  11. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 476
  12. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 419
  13. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 345
  14. 2016Top-down Neural Attention by Excitation Backpropcited by 321
  15. 2019Counterfactual Visual Explanationscited by 313
  16. 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 259
  17. 2015Explaining NonLinear Classification Decisions with Deep Taylor Decompositioncited by 250
  18. 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
  19. 2017Distilling a Neural Network Into a Soft Decision Treecited by 236
  20. 2017Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-supervised Object and Action Localizationcited by 207
  21. 2019Full-Gradient Representation for Neural Network Visualizationcited by 191
  22. 2015Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentationcited by 189
  23. 2017Visualizing Deep Neural Network Decisions: Prediction Difference Analysiscited by 166
  24. 2018Adversarial Complementary Learning for Weakly Supervised Object Localizationcited by 166
  25. 2020Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNscited by 151
  26. 2019Attention-based Dropout Layer for Weakly Supervised Object Localizationcited by 125
  27. 2019Large-scale interactive object segmentation with human annotatorscited by 110
  28. 2020Evaluating Weakly Supervised Object Localization Methods Rightcited by 105
  29. 2020Estimating Example Difficulty Using Variance of Gradientscited by 97
  30. 2017Learning how to explain neural networks: PatternNet and PatternAttributioncited by 84
  31. 2018Self-produced Guidance for Weakly-supervised Object Localizationcited by 72
  32. 2018NSML: Meet the MLaaS platform with a real-world case studycited by 70
  33. 2019Interpretability Beyond Classification Output: Semantic Bottleneck Networkscited by 49
  34. 2018Human-in-the-Loop Interpretability Priorcited by 36
  35. 2020Evaluation for Weakly Supervised Object Localization: Protocol, Metrics, and Datasetscited by 19
  36. 2020Self-explaining AI as an alternative to interpretable AIcited by 18
  37. 2020In-sample Contrastive Learning and Consistent Attention for Weakly Supervised Object Localizationcited by 16
  38. 2020Learning Propagation Rules for Attribution Map Generationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 2; 2024: 4; 2025: 2; 2026: 1.

15 papers.

Sort:

By area:

  1. 2022Large Loss Matters in Weakly Supervised Multi-Label Classificationcited by 38
  2. 2022Bridging the Gap between Classification and Localization for Weakly Supervised Object Localizationcited by 25
  3. 2022CLEVR-X: A Visual Reasoning Dataset for Natural Language Explanationscited by 16
  4. 2022Anti-Adversarially Manipulated Attributions for Weakly Supervised Semantic Segmentation and Object Localizationcited by 12
  5. 2022On Label Granularity and Object Localizationcited by 12
  6. 2023Bridging the Gap between Model Explanations in Partially Annotated Multi-label Classificationcited by 11
  7. 2024Feature Accentuation: Revealing 'What' Features Respond to in Natural Imagescited by 5
  8. 2023Embedding Space Interpolation Beyond Mini-Batch, Beyond Pairs and Beyond Examplescited by 4
  9. 2021Memory Regulation and Alignment toward Generalizer RGB-Infrared Personcited by 2
  10. 2024Improving Weakly-Supervised Object Localization Using Adversarial Erasing and Pseudo Labelcited by 2
  11. 2025A Survey on Interpretability in Visual Recognitioncited by 2
  12. 2025Towards Fine-Grained Interpretability: Counterfactual Explanations for Misclassification with Saliency Partitioncited by 2
3 more
  1. 2024Dual-Channel Reliable Breast Ultrasound Image Classification Based on Explainable Attribution and Uncertainty Quantificationcited by 0
  2. 2024Graphical Perception of Saliency-based Model Explanationscited by 0
  3. 2026Class Activation Mapping in Explainable Computer Vision: A Method-Centered Review of CNN, Transformer, and Foundation-Model-Era Visual Explanationscited by 0