mlsanityData to 7 Oct 2026

A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?

arXiv 2304.04780 ↗First posted 10 Apr 2023cs.LG cs.AIcited by 24

Not on the map. Why?

Abstract (arXiv)

Artificial intelligence (AI) models are increasingly finding applications in the field of medicine. Concerns have been raised about the explainability of the decisions that are made by these AI models. In this article, we give a systematic analysis of explainable artificial intelligence (XAI), with a primary focus on models that are currently being used in the field of healthcare. The literature search is conducted following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) standards for relevant work published from 1 January 2012 to 02 February 2022. The review analyzes the prevailing trends in XAI and lays out the major directions in which research is headed. We investigate the why, how, and when of the uses of these XAI models and their implications. We present a comprehensive examination of XAI methodologies as well as an explanation of how a trustworthy AI can be derived from describing AI models for healthcare fields. The discussion of this work will contribute to the formalization of the XAI field.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 70 of its 202 references that are in the map. First posted on arXiv 10 Apr 2023.

Builds on

70 of 202 references.

References · 70

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  6. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  7. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  8. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  9. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  10. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  11. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  12. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,182
38 more
  1. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  2. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  3. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  4. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 467
  5. 2019A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAIcited by 403
  6. 2017Deep Learning for Medical Image Analysiscited by 399
  7. 2016RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanismcited by 344
  8. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 342
  9. 2018Model compression via distillation and quantizationcited by 287
  10. 2018Scalable and accurate deep learning for electronic health recordscited by 275
  11. 2018Manipulating and Measuring Model Interpretabilitycited by 256
  12. 2020Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Surveycited by 252
  13. 2015Explaining NonLinear Classification Decisions with Deep Taylor Decompositioncited by 248
  14. 2017Distilling a Neural Network Into a Soft Decision Treecited by 230
  15. 2015Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction modelcited by 140
  16. 2018Explaining Explanations in AIcited by 136
  17. 2019What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Usecited by 134
  18. 2017What do we need to build explainable AI systems for the medical domain?cited by 127
  19. 2020The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategiescited by 91
  20. 2021Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyondcited by 75
  21. 2019An Evaluation of the Human-Interpretability of Explanationcited by 71
  22. 2017Beyond Sparsity: Tree Regularization of Deep Models for Interpretabilitycited by 68
  23. 2021Going Deeper through the Gleason Scoring Scale: An Automatic end-to-end System for Histology Prostate Grading and Cribriform Pattern Detectioncited by 66
  24. 2018RuleMatrix: Visualizing and Understanding Classifiers with Rulescited by 62
  25. 2018RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Recordscited by 61
  26. 2017MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Networkcited by 56
  27. 2017Interpretable & Explorable Approximations of Black Box Modelscited by 54
  28. 2018Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoningcited by 51
  29. 2019Benchmarking TPU, GPU, and CPU Platforms for Deep Learningcited by 51
  30. 2019Explainable artificial intelligence model to predict acute critical illness from electronic health recordscited by 42
  31. 2019"The Human Body is a Black Box": Supporting Clinical Decision-Making with Deep Learningcited by 39
  32. 2019U-CAM: Visual Explanation using Uncertainty based Class Activation Mapscited by 36
  33. 2020SurvLIME: A method for explaining machine learning survival modelscited by 34
  34. 2018Patient2Vec: A Personalized Interpretable Deep Representation of the Longitudinal Electronic Health Recordcited by 30
  35. 2021Semi-Supervised Variational Reasoning for Medical Dialogue Generationcited by 30
  36. 2019Evaluating Recurrent Neural Network Explanationscited by 29
  37. 2019Global and Local Interpretability for Cardiac MRI Classificationcited by 29
  38. 2019Explaining Models by Propagating Shapley Values of Local Componentscited by 29

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 5; 2025: 9; 2026: 5.

24 papers.

Sort:

  1. 2024Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Directioncited by 46
  2. 2025CoE: Chain-of-Explanation via Automatic Visual Concept Circuit Description and Polysemanticity Quantificationcited by 9
  3. 2025Investigating the Duality of Interpretability and Explainability in Machine Learningcited by 8
  4. 2023StyleCap: Automatic Speaking-Style Captioning from Speech Based on Speech and Language Self-supervised Learning Modelscited by 7
  5. 2023The Significance of Machine Learning in Clinical Disease Diagnosis: A Reviewcited by 4
  6. 2024QUCE: The Minimisation and Quantification of Path-Based Uncertainty for Generative Counterfactual Explanationscited by 4
  7. 2024Interpretable Machine Learning for Survival Analysiscited by 4
  8. 2025A Design Framework for operationalizing Trustworthy Artificial Intelligence in Healthcare: Requirements, Tradeoffs and Challenges for its Clinical Adoptioncited by 3
  9. 2024Explaining Human Activity Recognition with SHAP: Validating Insights with Perturbation and Quantitative Measurescited by 2
  10. 2025Modeling Quantum Machine Learning for Genomic Data Analysiscited by 2
  11. 2025Visual-Conversational Interface for Evidence-Based Explanation of Diabetes Risk Predictioncited by 2
  12. 2023Explainable AI in Orthopedics: Challenges, Opportunities, and Prospectscited by 1
12 more
  1. 2026An interpretable prototype parts-based neural network for medical tabular datacited by 1
  2. 2023CARNA: Characterizing Advanced heart failure Risk and hemodyNAmic phenotypes using learned multi-valued decision diagramscited by 0
  3. 2023Deciphering knee osteoarthritis diagnostic features with explainable artificial intelligence: A systematic reviewcited by 0
  4. 2024Choose Your Explanation: A Comparison of SHAP and GradCAM in Human Activity Recognitioncited by 0
  5. 2025Scaling Electronic Health Record Foundation Models for Population Health Managementcited by 0
  6. 2025ITL-LIME: Instance-Based Transfer Learning for Enhancing Local Explanations in Low-Resource Data Settingscited by 0
  7. 2025Governable AI: Provable Safety Under Extreme Threat Modelscited by 0
  8. 2025An Integrated Fusion Framework for Ensemble Learning Leveraging Gradient Boosting and Fuzzy Rule-Based Modelscited by 0
  9. 2026GradCFA: A Hybrid Gradient-Based Counterfactual and Feature Attribution Explanation Algorithm for Local Interpretation of Neural Networkscited by 0
  10. 2026Modeling Patient Care Trajectories with Transformer Hawkes Processescited by 0
  11. 2026NEURON: A Neuro-symbolic System for Grounded Clinical Explainabilitycited by 0
  12. 2026A Step Towards Inherently Interpretable Causal Machine Learning Models For Decision Supportcited by 0