mlsanityData to 14 Oct 2026

Designing Interpretable ML System to Enhance Trust in Healthcare: A Systematic Review to Proposed Responsible Clinician-AI-Collaboration Framework

arXiv 2311.11055 ↗First posted 18 Nov 2023cs.AI cs.HC cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

This paper explores the significant impact of AI-based medical devices, including wearables, telemedicine, large language models, and digital twins, on clinical decision support systems. It emphasizes the importance of producing outcomes that are not only accurate but also interpretable and understandable to clinicians, addressing the risk that lack of interpretability poses in terms of mistrust and reluctance to adopt these technologies in healthcare. The paper reviews interpretable AI processes, methods, applications, and the challenges of implementation in healthcare, focusing on quality control to facilitate responsible communication between AI systems and clinicians. It breaks down the interpretability process into data pre-processing, model selection, and post-processing, aiming to foster a comprehensive understanding of the crucial role of a robust interpretability approach in healthcare and to guide future research in this area. with insights for creating responsible clinician-AI tools for healthcare, as well as to offer a deeper understanding of the challenges they might face. Our research questions, eligibility criteria and primary goals were identified using Preferred Reporting Items for Systematic reviews and Meta-Analyses guideline and PICO method; PubMed, Scopus and Web of Science databases were systematically searched using sensitive and specific search strings. In the end, 52 publications were selected for data extraction which included 8 existing reviews and 44 related experimental studies. The paper offers general concepts of interpretable AI in healthcare and discuss three-levels interpretability process. Additionally, it provides a comprehensive discussion of evaluating robust interpretability AI in healthcare. Moreover, this survey introduces a step-by-step roadmap for implementing responsible AI in healthcare.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 25 of its 147 references that are in the map. First posted on arXiv 18 Nov 2023.

Builds on

25 of 147 references.

References · 25

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  2. 2018Datasheets for Datasetscited by 1,386
  3. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  4. 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
  5. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 443
  6. 2019A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAIcited by 404
  7. 2019Interpretable machine learning: definitions, methods, and applicationscited by 305
  8. 2016Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Modelscited by 213
  9. 2019Explainable Machine Learning in Deploymentcited by 211
  10. 2018Multimodal Explanations: Justifying Decisions and Pointing to the Evidencecited by 179
  11. 2020Drug discovery with explainable artificial intelligencecited by 118
  12. 2019Artificial Intelligence: the global landscape of ethics guidelinescited by 107
13 more
  1. 2021Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyondcited by 76
  2. 2021IAIA-BL: A Case-based Interpretable Deep Learning Model for Classification of Mass Lesions in Digital Mammographycited by 72
  3. 2023Towards Interpretable Mental Health Analysis with Large Language Modelscited by 53
  4. 2018TED: Teaching AI to Explain its Decisionscited by 33
  5. 2021EXplainable Neural-Symbolic Learning (X-NeSyL) methodology to fuse deep learning representations with expert knowledge graphs: the MonuMAI cultural heritage use casecited by 30
  6. 2022Explainable AI for clinical and remote health applications: a survey on tabular and time series datacited by 24
  7. 2021Sibyl: Understanding and Addressing the Usability Challenges of Machine Learning In High-Stakes Decision Makingcited by 19
  8. 2018Deep Weighted Averaging Classifierscited by 9
  9. 2023Missing Values and Imputation in Healthcare Data: Can Interpretable Machine Learning Help?cited by 6
  10. 2022A novel interpretable machine learning system to generate clinical risk scores: An application for predicting early mortality or unplanned readmission in a retrospective cohort studycited by 3
  11. 2019Parallel Coordinate Order for High-Dimensional Datacited by 2
  12. 2021A New Approach for Interpretability and Reliability in Clinical Risk Prediction: Acute Coronary Syndrome Scenariocited by 1
  13. 2023AI Framework for Early Diagnosis of Coronary Artery Disease: An Integration of Borderline SMOTE, Autoencoders and Convolutional Neural Networks Approachcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 4.

11 papers.

  1. 2024Large Language Models in Biomedical and Health Informatics: A Review with Bibliometric Analysiscited by 10
  2. 2024LLM-Agent-UMF: LLM-based Agent Unified Modeling Framework for Seamless Design of Multi Active/Passive Core-Agent Architecturescited by 5
  3. 2025A Design Framework for operationalizing Trustworthy Artificial Intelligence in Healthcare: Requirements, Tradeoffs and Challenges for its Clinical Adoptioncited by 3
  4. 2025Ontology- and LLM-based Data Harmonization for Federated Learning in Healthcarecited by 3
  5. 2025Making Sense of AI Limitations: How Individual Perceptions Shape Organizational Readiness for AI Adoptioncited by 2
  6. 2025Interpretable Multimodal Learning for Tumor Protein-Metal Binding: Progress, Challenges, and Perspectivescited by 1
  7. 2026Learning Label-Efficient Interpretable Medical Image Diagnosis via Semi-supervised Hypergraph Concept Bottleneck Modelcited by 1
  8. 2025Bridging AI Innovation and Healthcare Needs: Lessons Learned from Incorporating Modern NLP at The BC Cancer Registrycited by 0
  9. 2026Self-Explainability in Self-Adaptive and Self-Organising Systems: Status and Research Directionscited by 0
  10. 2026TraMP-LLaMA: Generative Interpretability with Decoupled Instruction Tuning for Facial Expression Quality Assessmentcited by 0
  11. 2026SIFTING: A Novel LLM-Based Framework for Structured and Transparent Information Extraction from Clinical Free-Text Reports, with Application to Tumor Staging in Lung Cancercited by 0