U-CAM: Visual Explanation using Uncertainty based Class Activation Maps
Not on the map. Why?
Abstract (arXiv)
Understanding and explaining deep learning models is an imperative task. Towards this, we propose a method that obtains gradient-based certainty estimates that also provide visual attention maps. Particularly, we solve for visual question answering task. We incorporate modern probabilistic deep learning methods that we further improve by using the gradients for these estimates. These have two-fold benefits: a) improvement in obtaining the certainty estimates that correlate better with misclassified samples and b) improved attention maps that provide state-of-the-art results in terms of correlation with human attention regions. The improved attention maps result in consistent improvement for various methods for visual question answering. Therefore, the proposed technique can be thought of as a recipe for obtaining improved certainty estimates and explanation for deep learning models. We provide detailed empirical analysis for the visual question answering task on all standard benchmarks and comparison with state of the art methods.
Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Aug 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 6; 2022: 7; 2023: 13; 2024: 6; 2025: 0; 2026: 4.
36 papers.
Sort:
By area:
- 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
- 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
- 2022Grad-SAM: Explaining Transformers via Gradient Self-Attention Mapscited by 42
- 2022Guidelines and Evaluation of Clinical Explainable AI in Medical Image Analysiscited by 35
- 2023The Treasure Beneath Multiple Annotations: An Uncertainty-aware Edge Detectorcited by 25
- 2023Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directionscited by 25
- 2023A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?cited by 24
- 2021Algorithm Fairness in AI for Medicine and Healthcarecited by 21
- 2022Editing Out-of-domain GAN Inversion via Differential Activationscited by 14
- 2021Do Not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgettingcited by 12
- 2022Joint Answering and Explanation for Visual Commonsense Reasoningcited by 11
- 2023S3C: Semi-Supervised VQA Natural Language Explanation via Self-Critical Learningcited by 10
24 more
- 2024PaPr: Training-Free One-Step Patch Pruning with Lightweight ConvNets for Faster Inferencecited by 9
- 2023Sentence Attention Blocks for Answer Groundingcited by 8
- 2022Boundary-Aware Uncertainty for Feature Attribution Explainerscited by 6
- 2023Flexible Visual Recognition by Evidential Modeling of Confusion and Ignorancecited by 6
- 2023Empowering CAM-Based Methods with Capability to Generate Fine-Grained and High-Faithfulness Explanationscited by 5
- 2023Towards More Faithful Natural Language Explanation Using Multi-Level Contrastive Learning in VQAcited by 5
- 2021On the use of uncertainty in classifying Aedes Albopictus mosquitoescited by 4
- 2021Explainable, automated urban interventions to improve pedestrian and vehicle safetycited by 3
- 2022On the Evaluation of the Plausibility and Faithfulness of Sentiment Analysis Explanationscited by 3
- 2023Information based explanation methods for deep learning agents -- with applications on large open-source chess modelscited by 3
- 2024VOICE: Variance of Induced Contrastive Explanations to quantify Uncertainty in Neural Network Interpretabilitycited by 3
- 2021Generating and Evaluating Explanations of Attended and Error-Inducing Input Regions for VQA Modelscited by 2
- 2024Generalizing GradCAM for Embedding Networkscited by 2
- 2026Inference Attacks Against Graph Generative Diffusion Modelscited by 2
- 2023Uncertainty Quantification for Local Model Explanations Without Model Accesscited by 1
- 2023Towards a Praxis for Intercultural Ethics in Explainable AIcited by 1
- 2023On Model Explanations with Transferable Neural Pathwayscited by 1
- 2024Applications of Explainable artificial intelligence in Earth system sciencecited by 1
- 2026Explanation-Guided Adversarial Training for Robust and Interpretable Modelscited by 1
- 2021Unsupervised Embedding Learning from Uncertainty Momentum Modelingcited by 0
- 2023CEnt: An Entropy-based Model-agnostic Explainability Framework to Contrast Classifiers' Decisionscited by 0
- 2024Segmentation of Non-Small Cell Lung Carcinomas: Introducing DRU-Net and Multi-Lens Distortioncited by 0
- 2026No Single Metric Tells the Whole Story: A Multi-Dimensional Evaluation Framework for Uncertainty Attributionscited by 0
- 2026Signal or Noise? Auditing Rotation-Induced Saliency Drift in Medical and Aerial Imagingcited by 0
- 2026Signal or Noise? Auditing Rotation-Induced Saliency Drift in Medical and Aerial Imaging2 Sep 2026
- 2026No Single Metric Tells the Whole Story: A Multi-Dimensional Evaluation Framework for Uncertainty Attributions25 Mar 2026
- 2026Explanation-Guided Adversarial Training for Robust and Interpretable Models2 Mar 2026
- 2026Inference Attacks Against Graph Generative Diffusion Models7 Jan 2026
- 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey3 Dec 2024
- 2024Segmentation of Non-Small Cell Lung Carcinomas: Introducing DRU-Net and Multi-Lens Distortion20 Jun 2024
- 2024Applications of Explainable artificial intelligence in Earth system science12 Jun 2024
- 2024VOICE: Variance of Induced Contrastive Explanations to quantify Uncertainty in Neural Network Interpretability1 Jun 2024
- 2024PaPr: Training-Free One-Step Patch Pruning with Lightweight ConvNets for Faster Inference24 Mar 2024
- 2024Generalizing GradCAM for Embedding Networks1 Feb 2024
- 2023Towards More Faithful Natural Language Explanation Using Multi-Level Contrastive Learning in VQA21 Dec 2023
- 2023Sentence Attention Blocks for Answer Grounding20 Sep 2023
24 more
- 2023On Model Explanations with Transferable Neural Pathways18 Sep 2023
- 2023Information based explanation methods for deep learning agents -- with applications on large open-source chess models18 Sep 2023
- 2023Flexible Visual Recognition by Evidential Modeling of Confusion and Ignorance14 Sep 2023
- 2023S3C: Semi-Supervised VQA Natural Language Explanation via Self-Critical Learning5 Sep 2023
- 2023Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directions9 Jun 2023
- 2023Towards a Praxis for Intercultural Ethics in Explainable AI24 Apr 2023
- 2023A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?10 Apr 2023
- 2023The Treasure Beneath Multiple Annotations: An Uncertainty-aware Edge Detector21 Mar 2023
- 2023Empowering CAM-Based Methods with Capability to Generate Fine-Grained and High-Faithfulness Explanations16 Mar 2023
- 2023CEnt: An Entropy-based Model-agnostic Explainability Framework to Contrast Classifiers' Decisions19 Jan 2023
- 2023Uncertainty Quantification for Local Model Explanations Without Model Access13 Jan 2023
- 2022On the Evaluation of the Plausibility and Faithfulness of Sentiment Analysis Explanations13 Oct 2022
- 2022Boundary-Aware Uncertainty for Feature Attribution Explainers5 Oct 2022
- 2022Editing Out-of-domain GAN Inversion via Differential Activations17 Jul 2022
- 2022Grad-SAM: Explaining Transformers via Gradient Self-Attention Maps23 Apr 2022
- 2022Joint Answering and Explanation for Visual Commonsense Reasoning25 Feb 2022
- 2022Guidelines and Evaluation of Clinical Explainable AI in Medical Image Analysis16 Feb 2022
- 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI20 Jan 2022
- 2021On the use of uncertainty in classifying Aedes Albopictus mosquitoes29 Oct 2021
- 2021Explainable, automated urban interventions to improve pedestrian and vehicle safety22 Oct 2021
- 2021Algorithm Fairness in AI for Medicine and Healthcare1 Oct 2021
- 2021Unsupervised Embedding Learning from Uncertainty Momentum Modeling19 Jul 2021
- 2021Generating and Evaluating Explanations of Attended and Error-Inducing Input Regions for VQA Models26 Mar 2021
- 2021Do Not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgetting3 Feb 2021
- 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
- 2022Grad-SAM: Explaining Transformers via Gradient Self-Attention Mapscited by 42
- 2022Guidelines and Evaluation of Clinical Explainable AI in Medical Image Analysiscited by 35
- 2023The Treasure Beneath Multiple Annotations: An Uncertainty-aware Edge Detectorcited by 25
- 2023Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directionscited by 25
- 2023A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?cited by 24
- 2021Algorithm Fairness in AI for Medicine and Healthcarecited by 21
- 2022Editing Out-of-domain GAN Inversion via Differential Activationscited by 14
- 2021Do Not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgettingcited by 12
- 2023S3C: Semi-Supervised VQA Natural Language Explanation via Self-Critical Learningcited by 10
- 2024PaPr: Training-Free One-Step Patch Pruning with Lightweight ConvNets for Faster Inferencecited by 9
- 2023Sentence Attention Blocks for Answer Groundingcited by 8
12 most cited of 33 All 33 (JSON)
- 2026Signal or Noise? Auditing Rotation-Induced Saliency Drift in Medical and Aerial Imaging2 Sep 2026
- 2026No Single Metric Tells the Whole Story: A Multi-Dimensional Evaluation Framework for Uncertainty Attributions25 Mar 2026
- 2026Explanation-Guided Adversarial Training for Robust and Interpretable Models2 Mar 2026
- 2024Segmentation of Non-Small Cell Lung Carcinomas: Introducing DRU-Net and Multi-Lens Distortion20 Jun 2024
- 2024Applications of Explainable artificial intelligence in Earth system science12 Jun 2024
- 2024VOICE: Variance of Induced Contrastive Explanations to quantify Uncertainty in Neural Network Interpretability1 Jun 2024
- 2024PaPr: Training-Free One-Step Patch Pruning with Lightweight ConvNets for Faster Inference24 Mar 2024
- 2024Generalizing GradCAM for Embedding Networks1 Feb 2024
- 2023Towards More Faithful Natural Language Explanation Using Multi-Level Contrastive Learning in VQA21 Dec 2023
- 2023Sentence Attention Blocks for Answer Grounding20 Sep 2023
- 2023On Model Explanations with Transferable Neural Pathways18 Sep 2023
- 2023Information based explanation methods for deep learning agents -- with applications on large open-source chess models18 Sep 2023
12 newest of 33 All 33 (JSON)