mlsanityData to 14 Oct 2026

MRxaI: Black-Box Explainability for Image Classifiers in a Medical Setting

arXiv 2311.14471 ↗First posted 24 Nov 2023cs.CV cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Existing tools for explaining the output of image classifiers can be divided into white-box, which rely on access to the model internals, and black-box, agnostic to the model. As the usage of AI in the medical domain grows, so too does the usage of explainability tools. Existing work on medical image explanations focuses on white-box tools, such as gradcam. However, there are clear advantages to switching to a black-box tool, including the ability to use it with any classifier and the wide selection of black-box tools available. On standard images, black-box tools are as precise as white-box. In this paper we compare the performance of several black-box methods against gradcam on a brain cancer MRI dataset. We demonstrate that most black-box tools are not suitable for explaining medical image classifications and present a detailed analysis of the reasons for their shortcomings. We also show that one black-box tool, a causal explainability-based rex, performs as well as gradcam.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 18 of its 53 references that are in the map. First posted on arXiv 24 Nov 2023.

Builds on

18 of 53 references.

References · 18

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  6. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
  7. 2017Interpretation of Neural Networks is Fragilecited by 444
  8. 2020Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scanscited by 158
  9. 2021Explainable artificial intelligence (XAI) in deep learning-based medical image analysiscited by 147
  10. 2019Association of genomic subtypes of lower-grade gliomas with shape features automatically extracted by a deep learning algorithmcited by 77
  11. 2018A Review of Challenges and Opportunities in Machine Learning for Healthcited by 68
  12. 2023FUTURE-AI: International consensus guideline for trustworthy and deployable artificial intelligence in healthcarecited by 60
6 more
  1. 2021Explaining a Series of Models by Propagating Shapley Valuescited by 37
  2. 2020A Multilinear Sampling Algorithm to Estimate Shapley Valuescited by 33
  3. 2019Reproducibility in Machine Learning for Healthcited by 25
  4. 2022Evaluating Explainable AI on a Multi-Modal Medical Imaging Task: Can Existing Algorithms Fulfill Clinical Requirements?cited by 20
  5. 2022FAST-AID Brain: Fast and Accurate Segmentation Tool using Artificial Intelligence Developed for Braincited by 10
  6. 2019Controlling for Biasing Signals in Images for Prognostic Models: Survival Predictions for Lung Cancer with Deep Learningcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 2.

6 papers.

  1. 2024Causal Explanations for Image Classifierscited by 9
  2. 2025Sufficient, Necessary and Complete Causal Explanations in Image Classificationcited by 4
  3. 2025I Am Big, You Are Little; I Am Right, You Are Wrongcited by 3
  4. 2025SpecReX: Explainable AI for Raman Spectroscopycited by 1
  5. 2026Quantifying Explainable AI-introduced signal noise on ECG data with Spectral Entropycited by 0
  6. 2026Generating Medical Image Counterfactuals using Causal Explanationscited by 0