TextCAM: Explaining Class Activation Map with Text
Not on the map. Why?
Abstract (arXiv)
Deep neural networks (DNNs) have achieved remarkable success across domains but remain difficult to interpret, limiting their trustworthiness in high-stakes applications. This paper focuses on deep vision models, for which a dominant line of explainability methods are Class Activation Mapping (CAM) and its variants working by highlighting spatial regions that drive predictions. We figure out that CAM provides little semantic insight into what attributes underlie these activations. To address this limitation, we propose TextCAM, a novel explanation framework that enriches CAM with natural languages. TextCAM combines the precise spatial localization of CAM with the semantic alignment of vision-language models (VLMs). Specifically, we derive channel-level semantic representations using CLIP embeddings and linear discriminant analysis, and aggregate them with CAM weights to produce textual descriptions of salient visual evidence. This yields explanations that jointly specify where the model attends and what visual attributes likely support its decision. We further extend TextCAM to generate feature channels into semantically coherent groups, enabling more fine-grained visual-textual explanations. Experiments on ImageNet, CLEVR, and CUB demonstrate that TextCAM produces faithful and interpretable rationales that improve human understanding, detect spurious correlations, and preserve model fidelity.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 57 references that are in the map. First posted on arXiv 1 Oct 2025.
Builds on
43 of 57 references.
References · 43
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023GPT-4 Technical Reportcited by 18,396
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2023Visual Instruction Tuningcited by 7,869
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
31 more
- 2015Learning Deep Features for Discriminative Localizationcited by 2,436
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
- 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
- 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
- 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
- 2014Striving for Simplicity: The All Convolutional Netcited by 990
- 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 939
- 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
- 2017Counterfactual Fairnesscited by 804
- 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
- 2018Recognition in Terra Incognitacited by 658
- 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 582
- 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 577
- 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 436
- 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
- 2019A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAIcited by 404
- 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
- 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 300
- 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 259
- 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 255
- 2017Real Time Image Saliency for Black Box Classifierscited by 246
- 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 196
- 2020Eigen-CAM: Class Activation Map using Principal Componentscited by 189
- 2020Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNscited by 151
- 2018Explaining Image Classifiers by Counterfactual Generationcited by 144
- 2019Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Modelscited by 118
- 2019XRAI: Better Attributions Through Regionscited by 116
- 2020There and Back Again: Revisiting Backpropagation Saliency Methodscited by 60
- 2022Disentangling visual and written concepts in CLIPcited by 58
- 2025Finer-CAM: Spotting the Difference Reveals Finer Details for Visual Explanationcited by 15
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026TextTeacher: What Can Language Teach About Images?
- 2025Zero-Shot Textual Explanations via Translating Decision-Critical Features
- 2024GIFT: A Framework Towards Global Interpretable Faithful Textual Explanations of Vision Classifiers
- 2024TextCAVs: Debugging vision models using text
- 2025DEXTER: Diffusion-Guided EXplanations with TExtual Reasoning for Vision Models
- 2023Improved Visual Grounding through Self-Consistent Explanations
- 2024VALE: A Multimodal Visual and Language Explanation Framework for Image Classifiers using eXplainable AI and Language Models
- 2026DEX-AR: A Dynamic Explainability Method for Autoregressive Vision-Language Models