mlsanityData to 14 Oct 2026

Explainable AI applications in the Medical Domain: a systematic review

arXiv 2308.05411 ↗First posted 10 Aug 2023cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Artificial Intelligence in Medicine has made significant progress with emerging applications in medical imaging, patient care, and other areas. While these applications have proven successful in retrospective studies, very few of them were applied in practice.The field of Medical AI faces various challenges, in terms of building user trust, complying with regulations, using data ethically.Explainable AI (XAI) aims to enable humans understand AI and trust its results. This paper presents a literature review on the recent developments of XAI solutions for medical decision support, based on a representative sample of 198 articles published in recent years. The systematic synthesis of the relevant articles resulted in several findings. (1) model-agnostic XAI techniques were mostly employed in these solutions, (2) deep learning models are utilized more than other types of machine learning models, (3) explainability was applied to promote trust, but very few works reported the physicians participation in the loop, (4) visual and interactive user interface is more useful in understanding the explanation and the recommendation of the system. More research is needed in collaboration between medical and AI experts, that could guide the development of suitable frameworks for the design, implementation, and evaluation of XAI solutions in medicine.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 51 of its 165 references that are in the map. First posted on arXiv 10 Aug 2023.

Builds on

51 of 165 references.

References · 51

  1. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  3. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  4. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  5. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  6. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
  7. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  8. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  9. 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
  10. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
  11. 2016The Mythos of Model Interpretabilitycited by 721
  12. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 556
38 more
  1. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  2. 2017CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learningcited by 461
  3. 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 372
  4. 2016RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanismcited by 349
  5. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 345
  6. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 297
  7. 2018Metrics for Explainable AI: Challenges and Prospectscited by 260
  8. 2020Questioning the AI: Informing Design Practices for Explainable AI User Experiencescited by 253
  9. 2018Techniques for Interpretable Machine Learningcited by 238
  10. 2017Distilling a Neural Network Into a Soft Decision Treecited by 236
  11. 2017Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictionscited by 234
  12. 2017Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Modelscited by 206
  13. 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
  14. 2015Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction modelcited by 140
  15. 2017What do we need to build explainable AI systems for the medical domain?cited by 129
  16. 2020Explainable Deep Learning: A Field Guide for the Uninitiatedcited by 127
  17. 2020Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challengescited by 120
  18. 2020Explainable deep learning models in medical image analysiscited by 111
  19. 2019Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approachescited by 101
  20. 2017Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciencescited by 85
  21. 2015Supersparse Linear Integer Models for Optimized Medical Scoring Systemscited by 76
  22. 2017What Does Explainable AI Really Mean? A New Conceptualization of Perspectivescited by 72
  23. 2019Understanding Neural Networks via Feature Visualization: A surveycited by 65
  24. 2019Explainable AI for Trees: From Local Explanations to Global Understandingcited by 65
  25. 2018RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Recordscited by 61
  26. 2019Towards Explainable Deep Neural Networks (xDNN)cited by 61
  27. 2018DeepAffinity: Interpretable Deep Learning of Compound-Protein Affinity through Unified Recurrent and Convolutional Neural Networkscited by 52
  28. 2017Interpreting CNN Knowledge via an Explanatory Graphcited by 49
  29. 2018Explanation Methods in Deep Learning: Users, Values, Concerns and Challengescited by 49
  30. 2020The Emerging Landscape of Explainable AI Planning and Decision Makingcited by 44
  31. 2014Interpreting Tree Ensembles with inTreescited by 37
  32. 2016Making Tree Ensembles Interpretable: A Bayesian Model Selection Approachcited by 27
  33. 2021VBridge: Connecting the Dots Between Features and Data to Explain Healthcare Modelscited by 12
  34. 2019Incorporating Task-Specific Structural Knowledge into CNNs for Brain Midline Shift Detectioncited by 7
  35. 2018Collaborative Human-AI (CHAI): Evidence-Based Interpretable Melanoma Classification in Dermoscopic Imagescited by 4
  36. 2020Machine Reasoning Explainabilitycited by 4
  37. 2021Diagnosis of Acute Poisoning Using Explainable Artificial Intelligencecited by 2
  38. 2019Explainable Software for Cyber-Physical Systems (ES4CPS): Report from the GI Dagstuhl Seminar 19023, January 06-11 2019, Schloss Dagstuhlcited by 1

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 1.

5 papers.

  1. 2024DWARF: Disease-weighted network for attention map refinementcited by 5
  2. 2024AI Readiness in Healthcare through Storytelling XAIcited by 4
  3. 2024Understanding Neural Network Systems for Image Analysis using Vector Spaces and Inverse Mapscited by 1
  4. 2025Deep Learning for Ophthalmology: The State-of-the-Art and Future Trendscited by 1
  5. 2026A User-Centric Analysis of Explainability in AI-Based Medical Image Diagnosiscited by 0