# A Design Framework for operationalizing Trustworthy Artificial Intelligence in Healthcare: Requirements, Tradeoffs and Challenges for its Clinical Adoption arXiv: https://arxiv.org/abs/2504.19179 ยท HTML: https://mlsanity.com/paper/2504.19179/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Builds on 32 of its 151 references that are in the map. - First posted on arXiv 27 Apr 2025. ## Builds on (most cited first, 30 of 32) - [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016) - [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019) - [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017) - [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014) - [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016) - [Algorithmic decision making and the cost of fairness](https://mlsanity.com/paper/1701.08230/) (2017) - [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017) - [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022) - [Path-Specific Counterfactual Fairness](https://mlsanity.com/paper/1802.08139/) (2018) - [Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees](https://mlsanity.com/paper/1806.06055/) (2018) - [A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual & Group Unfairness via Inequality Indices](https://mlsanity.com/paper/1807.00787/) (2018) - [The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies](https://mlsanity.com/paper/2007.15911/) (2020) - [Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges](https://mlsanity.com/paper/2201.08135/) (2022) - [Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods](https://mlsanity.com/paper/2111.02398/) (2021) - [Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanations](https://mlsanity.com/paper/1912.09024/) (2019) - [Online Learning with an Unknown Fairness Metric](https://mlsanity.com/paper/1802.06936/) (2018) - [Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulation](https://mlsanity.com/paper/2305.02231/) (2023) - [Socially Responsible AI Algorithms: Issues, Purposes, and Challenges](https://mlsanity.com/paper/2101.02032/) (2021) - [Fairness Through Computationally-Bounded Awareness](https://mlsanity.com/paper/1803.03239/) (2018) - [Auditing Black-box Models for Indirect Influence](https://mlsanity.com/paper/1602.07043/) (2016) - [FAE: A Fairness-Aware Ensemble Framework](https://mlsanity.com/paper/2002.00695/) (2020) - [Unraveling the Hidden Environmental Impacts of AI Solutions for Environment](https://mlsanity.com/paper/2110.11822/) (2021) - [Handling missing values in healthcare data: A systematic review of deep learning-based imputation techniques](https://mlsanity.com/paper/2210.08258/) (2022) - [A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?](https://mlsanity.com/paper/2304.04780/) (2023) - [Physical Adversarial Attacks For Camera-based Smart Systems: Current Trends, Categorization, Applications, Research Challenges, and Future Outlook](https://mlsanity.com/paper/2308.06173/) (2023) - [Data Harmonisation for Information Fusion in Digital Healthcare: A State-of-the-Art Systematic Review, Meta-Analysis and Future Research Directions](https://mlsanity.com/paper/2201.06505/) (2022) - [Can I trust my fake data -- A comprehensive quality assessment framework for synthetic tabular data in healthcare](https://mlsanity.com/paper/2401.13716/) (2024) - [Trustworthy Federated Learning via Blockchain](https://mlsanity.com/paper/2209.04418/) (2022) - [Designing Interpretable ML System to Enhance Trust in Healthcare: A Systematic Review to Proposed Responsible Clinician-AI-Collaboration Framework](https://mlsanity.com/paper/2311.11055/) (2023) - [Holding AI to Account: Challenges for the Delivery of Trustworthy AI in Healthcare](https://mlsanity.com/paper/2211.16444/) (2022) ## Built on by (most cited first, 3 of 3) - [More Human or More AI? Visualizing Human-AI Collaboration Disclosures in Journalistic News Production](https://mlsanity.com/paper/2601.11072/) (2026) - [FairMed-XGB: A Bayesian-Optimised Multi-Metric Framework with Explainability for Demographic Equity in Critical Healthcare Data](https://mlsanity.com/paper/2603.14947/) (2026) - [YOLO with Kolmogorov-Arnold networks and vision-language foundation models for interpretable object detection with trustworthy multimodal AI in computer vision perception](https://mlsanity.com/paper/2603.23037/) (2026) ## Abstract (arXiv) Artificial Intelligence (AI) holds great promise for transforming healthcare, particularly in disease diagnosis, prognosis, and patient care. The increasing availability of digital medical data, such as images, omics, biosignals, and electronic health records, combined with advances in computing, has enabled AI models to approach expert-level performance. However, widespread clinical adoption remains limited, primarily due to challenges beyond technical performance, including ethical concerns, regulatory barriers, and lack of trust. To address these issues, AI systems must align with the principles of Trustworthy AI (TAI), which emphasize human agency and oversight, algorithmic robustness, privacy and data governance, transparency, bias and discrimination avoidance, and accountability. Yet, the complexity of healthcare processes (e.g., screening, diagnosis, prognosis, and treatment) and the diversity of stakeholders (clinicians, patients, providers, regulators) complicate the integration of TAI principles. To bridge the gap between TAI theory and practical implementation, this paper proposes a design framework to support developers in embedding TAI principles into medical AI systems. Thus, for each stakeholder identified across various healthcare processes, we propose a disease-agnostic collection of requirements that medical AI systems should incorporate to adhere to the principles of TAI. Additionally, we examine the challenges and tradeoffs that may arise when applying these principles in practice. To ground the discussion, we focus on cardiovascular diseases, a field marked by both high prevalence and active AI innovation, and demonstrate how TAI principles have been applied and where key obstacles persist.