# Domain aware medical image classifier interpretation by counterfactual impact analysis

arXiv: https://arxiv.org/abs/2007.06312 · HTML: https://mlsanity.com/paper/2007.06312/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jul 2020.

## Built on by (most cited first, 17 of 17)

- [Explainable artificial intelligence (XAI) in deep learning-based medical image analysis](https://mlsanity.com/paper/2107.10912/) (2021)
- [Deep Learning for Chest X-ray Analysis: A Survey](https://mlsanity.com/paper/2103.08700/) (2021)
- [Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods](https://mlsanity.com/paper/2111.02398/) (2021)
- [BiomedJourney: Counterfactual Biomedical Image Generation by Instruction-Learning from Multimodal Patient Journeys](https://mlsanity.com/paper/2310.10765/) (2023)
- [ICAM-reg: Interpretable Classification and Regression with Feature Attribution for Mapping Neurological Phenotypes in Individual Scans](https://mlsanity.com/paper/2103.02561/) (2021)
- [PredDiff: Explanations and Interactions from Conditional Expectations](https://mlsanity.com/paper/2102.13519/) (2021)
- [CaRTS: Causality-driven Robot Tool Segmentation from Vision and Kinematics Data](https://mlsanity.com/paper/2203.09475/) (2022)
- [An Interpretable Approach to Automated Severity Scoring in Pelvic Trauma](https://mlsanity.com/paper/2105.10238/) (2021)
- [Rethinking Causality-driven Robot Tool Segmentation with Temporal Constraints](https://mlsanity.com/paper/2212.00072/) (2022)
- [Multi-task fusion for improving mammography screening data classification](https://mlsanity.com/paper/2112.01320/) (2021)
- [Interpretable Medical Imagery Diagnosis with Self-Attentive Transformers: A Review of Explainable AI for Health Care](https://mlsanity.com/paper/2309.00252/) (2023)
- [Leveraging Conditional Generative Models in a General Explanation Framework of Classifier Decisions](https://mlsanity.com/paper/2106.10947/) (2021)
- [3D ReX: Causal Explanations in 3D Neuroimaging Classification](https://mlsanity.com/paper/2502.12181/) (2025)
- [Towards interpreting ML-based automated malware detection models: a survey](https://mlsanity.com/paper/2101.06232/) (2021)
- [Visual Interpretable and Explainable Deep Learning Models for Brain Tumor MRI and COVID-19 Chest X-ray Images](https://mlsanity.com/paper/2208.00953/) (2022)
- [eXplainable Artificial Intelligence on Medical Images: A Survey](https://mlsanity.com/paper/2305.07511/) (2023)
- [SSplain: Sparse and Smooth Explainer for Retinopathy of Prematurity Classification](https://mlsanity.com/paper/2512.08038/) (2025)

## Abstract (arXiv)

The success of machine learning methods for computer vision tasks has driven a surge in computer assisted prediction for medicine and biology. Based on a data-driven relationship between input image and pathological classification, these predictors deliver unprecedented accuracy. Yet, the numerous approaches trying to explain the causality of this learned relationship have fallen short: time constraints, coarse, diffuse and at times misleading results, caused by the employment of heuristic techniques like Gaussian noise and blurring, have hindered their clinical adoption. In this work, we discuss and overcome these obstacles by introducing a neural-network based attribution method, applicable to any trained predictor. Our solution identifies salient regions of an input image in a single forward-pass by measuring the effect of local image-perturbations on a predictor's score. We replace heuristic techniques with a strong neighborhood conditioned inpainting approach, avoiding anatomically implausible, hence adversarial artifacts. We evaluate on public mammography data and compare against existing state-of-the-art methods. Furthermore, we exemplify the approach's generalizability by demonstrating results on chest X-rays. Our solution shows, both quantitatively and qualitatively, a significant reduction of localization ambiguity and clearer conveying results, without sacrificing time efficiency.