mlsanityData to 7 Oct 2026

Federated learning, ethics, and the double black box problem in medical AI

arXiv 2504.20656 ↗First posted 29 Apr 2025cs.LG cs.AI cs.CYcited by 1

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is a machine learning approach that allows multiple devices or institutions to collaboratively train a model without sharing their local data with a third-party. FL is considered a promising way to address patient privacy concerns in medical artificial intelligence. The ethical risks of medical FL systems themselves, however, have thus far been underexamined. This paper aims to address this gap. We argue that medical FL presents a new variety of opacity -- federation opacity -- that, in turn, generates a distinctive double black box problem in healthcare AI. We highlight several instances in which the anticipated benefits of medical FL may be exaggerated, and conclude by highlighting key challenges that must be overcome to make FL ethically feasible in medicine.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 41 of its 133 references that are in the map. First posted on arXiv 29 Apr 2025.

Builds on

41 of 133 references.

References · 41

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  6. 2020Adaptive Federated Optimizationcited by 1,036
  7. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  8. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  9. 2020The Future of Digital Health with Federated Learningcited by 625
  10. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  11. 2019Local Model Poisoning Attacks to Byzantine-Robust Federated Learningcited by 479
  12. 2019Agnostic Federated Learningcited by 451
29 more
  1. 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
  2. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 285
  3. 2018Representer Point Selection for Explaining Deep Neural Networkscited by 187
  4. 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
  5. 2021Federated Learning for Smart Healthcare: A Surveycited by 182
  6. 2018Multi-Institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentationcited by 148
  7. 2020Multi-site fMRI Analysis Using Privacy-preserving Federated Learning and Domain Adaptation: ABIDE Resultscited by 122
  8. 2019BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learningcited by 115
  9. 2022Vertical Federated Learning: Concepts, Advances and Challengescited by 104
  10. 2021MedPerf: Open Benchmarking Platform for Medical Artificial Intelligence using Federated Evaluationcited by 88
  11. 2022Federated Learning Enables Big Data for Rare Cancer Boundary Detectioncited by 83
  12. 2019Patient Clustering Improves Efficiency of Federated Machine Learning to predict mortality and hospital stay time using distributed Electronic Medical Recordscited by 70
  13. 2020Federated Learning for Breast Density Classification: A Real-World Implementationcited by 62
  14. 2021Collaborative Federated Learning For Healthcare: Multi-Modal COVID-19 Diagnosis at the Edgecited by 56
  15. 2021Incentive Mechanisms for Federated Learning: From Economic and Game Theoretic Perspectivecited by 47
  16. 2023The Unfairness of Fair Machine Learning: Levelling down and strict egalitarianism by defaultcited by 36
  17. 2017Federated Tensor Factorization for Computational Phenotypingcited by 20
  18. 2021Multi-task Federated Learning for Heterogeneous Pancreas Segmentationcited by 19
  19. 2016"Influence Sketching": Finding Influential Samples In Large-Scale Regressionscited by 17
  20. 2020Automated Pancreas Segmentation Using Multi-institutional Collaborative Deep Learningcited by 11
  21. 2023Federated Learning of Medical Concepts Embedding using BEHRTcited by 11
  22. 2023Recent Methodological Advances in Federated Learning for Healthcarecited by 11
  23. 2025Limits of trust in medical AIcited by 7
  24. 2023Efficient Classification of SARS-CoV-2 Spike Sequences Using Federated Learningcited by 4
  25. 2025A moving target in AI-assisted decision-making: Dataset shift, model updating, and the problem of update opacitycited by 3
  26. 2025Diachronic and synchronic variation in the performance of adaptive machine learning systems: The ethical challengescited by 3
  27. 2025High hopes for "Deep Medicine"? AI, economics, and the future of carecited by 3
  28. 2023Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Modelscited by 1
  29. 2024Implementing a Nordic-Baltic Federated Health Data Network: a case reportcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Deep Learning-based Assessment of the Relation Between the Third Molar and Mandibular Canal on Panoramic Radiographs using Local, Centralized, and Federated Learningcited by 0