mlsanityData to 7 Oct 2026

Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sources

arXiv 2305.06217 ↗First posted 10 May 2023cs.LG cs.AI cs.CRcited by 10

Not on the map. Why?

Abstract (arXiv)

Machine learning (ML) in healthcare presents numerous opportunities for enhancing patient care, population health, and healthcare providers' workflows. However, the real-world clinical and cost benefits remain limited due to challenges in data privacy, heterogeneous data sources, and the inability to fully leverage multiple data modalities. In this perspective paper, we introduce "patchwork learning" (PL), a novel paradigm that addresses these limitations by integrating information from disparate datasets composed of different data modalities (e.g., clinical free-text, medical images, omics) and distributed across separate and secure sites. PL allows the simultaneous utilization of complementary data sources while preserving data privacy, enabling the development of more holistic and generalizable ML models. We present the concept of patchwork learning and its current implementations in healthcare, exploring the potential opportunities and applicable data sources for addressing various healthcare challenges. PL leverages bridging modalities or overlapping feature spaces across sites to facilitate information sharing and impute missing data, thereby addressing related prediction tasks. We discuss the challenges associated with PL, many of which are shared by federated and multimodal learning, and provide recommendations for future research in this field. By offering a more comprehensive approach to healthcare data integration, patchwork learning has the potential to revolutionize the clinical applicability of ML models. This paradigm promises to strike a balance between personalization and generalizability, ultimately enhancing patient experiences, improving population health, and optimizing healthcare providers' workflows.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 80 of its 187 references that are in the map. First posted on arXiv 10 May 2023.

Builds on

80 of 187 references.

References · 80

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  6. 2019Invariant Risk Minimizationcited by 1,832
  7. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  8. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  9. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  10. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  11. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  12. 2021Domain Generalization: A Surveycited by 760
38 more
  1. 2017Federated Multi-Task Learningcited by 669
  2. 2019Attention is not not Explanationcited by 629
  3. 2020The Future of Digital Health with Federated Learningcited by 625
  4. 2013Domain Generalization via Invariant Feature Representationcited by 622
  5. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  6. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  7. 2020Learning under Concept Drift: A Reviewcited by 470
  8. 2021Towards Personalized Federated Learningcited by 462
  9. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  10. 2019Agnostic Federated Learningcited by 451
  11. 2019Fair Resource Allocation in Federated Learningcited by 389
  12. 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 340
  13. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 326
  14. 2020The Risks of Invariant Risk Minimizationcited by 262
  15. 2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 243
  16. 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
  17. 2018Multimodal Explanations: Justifying Decisions and Pointing to the Evidencecited by 179
  18. 2018Learning Factorized Multimodal Representationscited by 171
  19. 2018Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalitiescited by 149
  20. 2018Domain Generalization via Conditional Invariant Representationcited by 146
  21. 2019One-Shot Federated Learningcited by 124
  22. 2022FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settingscited by 121
  23. 2020Distilled One-Shot Federated Learningcited by 108
  24. 2016Towards the Science of Security and Privacy in Machine Learningcited by 89
  25. 2016Joint Multimodal Learning with Deep Generative Modelscited by 87
  26. 2022Integrated multimodal artificial intelligence framework for healthcare applicationscited by 69
  27. 2022Causal Machine Learning for Healthcare and Precision Medicinecited by 68
  28. 2021Towards Fair Federated Learning with Zero-Shot Data Augmentationcited by 64
  29. 2021Collaborative Federated Learning For Healthcare: Multi-Modal COVID-19 Diagnosis at the Edgecited by 56
  30. 2018Do Explanations make VQA Models more Predictable to a Human?cited by 51
  31. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
  32. 2019Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalizationcited by 43
  33. 2020An Overview of Privacy in Machine Learningcited by 39
  34. 2020A Reputation Mechanism Is All You Need: Collaborative Fairness and Adversarial Robustness in Federated Learningcited by 38
  35. 2022On the Power of Foundation Modelscited by 37
  36. 2020Dynamic Fusion based Federated Learning for COVID-19 Detectioncited by 36
  37. 2021Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directionscited by 35
  38. 2022Federated Learning under Distributed Concept Driftcited by 35

50 most cited of 80 All 80 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 1; 2026: 4.

10 papers.

  1. 2023Multimodal Federated Learning in Healthcare: a Reviewcited by 13
  2. 2024FairEHR-CLP: Towards Fairness-Aware Clinical Predictions with Contrastive Learning in Multimodal Electronic Health Recordscited by 6
  3. 2023Generating Medical Prescriptions with Conditional Transformercited by 3
  4. 2024Transfer Learning with Clinical Concept Embeddings from Large Language Modelscited by 2
  5. 2025A Framework for Situating Innovations, Opportunities, and Challenges in Advancing Vertical Systems with Large AI Modelscited by 2
  6. 2026MedPMC: A Systematic Framework for Scaling High-Fidelity Medical Multimodal Data for Foundation Modelscited by 1
  7. 2024Joint Analysis of Single-Cell Data across Cohorts with Missing Modalitiescited by 0
  8. 2026Efficient Imputation for Patch-based Missing Single-cell Data via Cluster-regularized Optimal Transportcited by 0
  9. 2026FeTTL: Federated Template and Task Learning for Multi-Institutional Medical Imagingcited by 0
  10. 2026ADAPTOOD: Uncertainty-Aware Fine-Tuning for Out-of-Distribution ECG Time Series Modelscited by 0