mlsanityData to 7 Oct 2026

Extracting Causal Visual Features for Limited label Classification

arXiv 2103.12322 ↗First posted 23 Mar 2021cs.CV cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Neural networks trained to classify images do so by identifying features that allow them to distinguish between classes. These sets of features are either causal or context dependent. Grad-CAM is a popular method of visualizing both sets of features. In this paper, we formalize this feature divide and provide a methodology to extract causal features from Grad-CAM. We do so by defining context features as those features that allow contrast between predicted class and any contrast class. We then apply a set theoretic approach to separate causal from contrast features for COVID-19 CT scans. We show that on average, the image regions with the proposed causal features require 15% less bits when encoded using Huffman encoding, compared to Grad-CAM, for an average increase of 3% classification accuracy, over Grad-CAM. Moreover, we validate the transfer-ability of causal features between networks and comment on the non-human interpretable causal nature of current networks.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 16 of its 25 references that are in the map. First posted on arXiv 23 Mar 2021.

Builds on

16 of 25 references.

References · 16

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2016Densely Connected Convolutional Networkscited by 6,691
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  6. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 936
  7. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  8. 2020COVID-CT-Dataset: A CT Scan Dataset about COVID-19cited by 93
  9. 2016Discovering Causal Signals in Imagescited by 79
  10. 2020Gradients as a Measure of Uncertainty in Neural Networkscited by 55
  11. 2020Backpropagated Gradient Representations for Anomaly Detectioncited by 54
  12. 2019A Machine Learning Benchmark for Facies Classificationcited by 27
4 more
  1. 2020Contrastive Explanations in Neural Networkscited by 26
  2. 2019Distorted Representation Space Characterization Through Backpropagated Gradientscited by 12
  3. 2020Implicit Saliency in Deep Neural Networkscited by 9
  4. 2019Relative Afferent Pupillary Defect Screening through Transfer Learningcited by 8

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 3; 2024: 4.

12 papers.

  1. 2022OLIVES Dataset: Ophthalmic Labels for Investigating Visual Eye Semanticscited by 23
  2. 2022Introspective Learning : A Two-Stage Approach for Inference in Neural Networkscited by 19
  3. 2022Gradient-Based Adversarial and Out-of-Distribution Detectioncited by 14
  4. 2022Explanatory Paradigms in Neural Networkscited by 11
  5. 2022Patient Aware Active Learning for Fine-Grained OCT Classificationcited by 9
  6. 2023Probing the Purview of Neural Networks via Gradient Analysiscited by 9
  7. 2024VOICE: Variance of Induced Contrastive Explanations to quantify Uncertainty in Neural Network Interpretabilitycited by 3
  8. 2023Stochastic Surprisal: An inferential measurement of Free Energy in Neural Networkscited by 2
  9. 2024Counterfactual Gradients-based Quantification of Prediction Trust in Neural Networkscited by 2
  10. 2024Are Objective Explanatory Evaluation metrics Trustworthy? An Adversarial Analysiscited by 2
  11. 2023Learning Trajectory-Conditioned Relations to Predict Pedestrian Crossing Behaviorcited by 0
  12. 2024Explaining Representation Learning with Perceptual Componentscited by 0