mlsanityData to 7 Oct 2026

FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings

arXiv 2210.04620 ↗First posted 10 Oct 2022cs.LG cs.CVcited by 121

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a novel approach enabling several clients holding sensitive data to collaboratively train machine learning models, without centralizing data. The cross-silo FL setting corresponds to the case of few (2--50) reliable clients, each holding medium to large datasets, and is typically found in applications such as healthcare, finance, or industry. While previous works have proposed representative datasets for cross-device FL, few realistic healthcare cross-silo FL datasets exist, thereby slowing algorithmic research in this critical application. In this work, we propose a novel cross-silo dataset suite focused on healthcare, FLamby (Federated Learning AMple Benchmark of Your cross-silo strategies), to bridge the gap between theory and practice of cross-silo FL. FLamby encompasses 7 healthcare datasets with natural splits, covering multiple tasks, modalities, and data volumes, each accompanied with baseline training code. As an illustration, we additionally benchmark standard FL algorithms on all datasets. Our flexible and modular suite allows researchers to easily download datasets, reproduce results and re-use the different components for their research. FLamby is available at~urlwww.github.com/owkin/flamby.

Key facts. Cited by 121 papers in the map (data through 7 Oct 2026). Builds on 68 of its 170 references that are in the map. First posted on arXiv 10 Oct 2022.

Builds on

68 of 170 references.

References · 68

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  8. 2020Scaling Laws for Neural Language Modelscited by 6,411
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  11. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  12. 2019Advances and Open Problems in Federated Learningcited by 3,021
38 more
  1. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  2. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  3. 2016Deep Learning with Differential Privacycited by 2,682
  4. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  6. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  7. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  8. 2018Attention-based Deep Multiple Instance Learningcited by 1,077
  9. 2020Adaptive Federated Optimizationcited by 1,036
  10. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  11. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  12. 2019Towards Federated Learning at Scale: System Designcited by 826
  13. 2018LEAF: A Benchmark for Federated Settingscited by 780
  14. 2017Focal Loss for Dense Object Detectioncited by 749
  15. 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 738
  16. 2020The Future of Digital Health with Federated Learningcited by 625
  17. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  18. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  19. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  20. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
  21. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  22. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
  23. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
  24. 2019BCN20000: Dermoscopic Lesions in the Wildcited by 224
  25. 2020A survey of loss functions for semantic segmentationcited by 216
  26. 2016Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challengecited by 198
  27. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
  28. 2019The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challengecited by 188
  29. 2020TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learningcited by 176
  30. 2019The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomescited by 168
  31. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  32. 2018Multi-Institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentationcited by 148
  33. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 135
  34. 2018Protection Against Reconstruction and Its Applications in Private Federated Learningcited by 127
  35. 2020Federated Visual Classification with Real-World Data Distributioncited by 126
  36. 2021FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Taskscited by 84
  37. 2019Learning with minibatch Wasserstein : asymptotic and gradient propertiescited by 83
  38. 2021On Large-Cohort Training for Federated Learningcited by 73

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 33; 2024: 32; 2025: 29; 2026: 27.

121 papers.

Sort:

By area:

  1. 2023FedMultimodal: A Benchmark For Multimodal Federated Learningcited by 51
  2. 2023Federated Conformal Predictors for Distributed Uncertainty Quantificationcited by 46
  3. 2023Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learningcited by 29
  4. 2023Think Twice Before Selection: Federated Evidential Active Learning for Medical Image Analysis with Domain Shiftscited by 26
  5. 2024FedLLM-Bench: Realistic Benchmarks for Federated Learning of Large Language Modelscited by 26
  6. 2023A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacycited by 24
  7. 2024Advances in APPFL: A Comprehensive and Extensible Federated Learning Frameworkcited by 19
  8. 2023FedGT: Identification of Malicious Clients in Federated Learning with Secure Aggregationcited by 16
  9. 2024Redefining Contributions: Shapley-Driven Federated Learningcited by 16
  10. 2024From Challenges and Pitfalls to Recommendations and Opportunities: Implementing Federated Learning in Healthcarecited by 16
  11. 2023Benchmarking Algorithms for Federated Domain Generalizationcited by 14
  12. 2023FedCompass: Efficient Cross-Silo Federated Learning on Heterogeneous Client Devices using a Computing Power Aware Schedulercited by 13
38 more
  1. 2023Multimodal Federated Learning in Healthcare: a Reviewcited by 13
  2. 2024Federated Learning with Bilateral Curation for Partially Class-Disjoint Datacited by 13
  3. 2024Revisiting Ensembling in One-Shot Federated Learningcited by 13
  4. 2023Bias Propagation in Federated Learningcited by 12
  5. 2023FedWon: Triumphing Multi-domain Federated Learning Without Normalizationcited by 11
  6. 2023FedNoisy: Federated Noisy Label Learning Benchmarkcited by 11
  7. 2023Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sourcescited by 10
  8. 2023ULDP-FL: Federated Learning with Across Silo User-Level Differential Privacycited by 9
  9. 2024Personalized Collaborative Fine-Tuning for On-Device Large Language Modelscited by 9
  10. 2023Collaborative Learning via Prediction Consensuscited by 8
  11. 2023FedAIoT: A Federated Learning Benchmark for Artificial Intelligence of Thingscited by 8
  12. 2023Handling Data Heterogeneity via Architectural Design for Federated Visual Recognitioncited by 8
  13. 2024A Systematic Review of Generalization Research in Medical Image Classificationcited by 8
  14. 2024pfl-research: simulation framework for accelerating research in Private Federated Learningcited by 8
  15. 2024FEDMEKI: A Benchmark for Scaling Medical Foundation Models via Federated Knowledge Injectioncited by 8
  16. 2023Federated Survival Forestscited by 7
  17. 2023FeSViBS: Federated Split Learning of Vision Transformer with Block Samplingcited by 7
  18. 2024Cross-silo Federated Learning with Record-level Personalized Differential Privacycited by 7
  19. 2024COALA: A Practical and Vision-Centric Federated Learning Platformcited by 7
  20. 2023Principled Federated Domain Adaptation: Gradient Projection and Auto-Weightingcited by 6
  21. 2023FedAds: A Benchmark for Privacy-Preserving CVR Estimation with Vertical Federated Learningcited by 6
  22. 2023Scaling Survival Analysis in Healthcare with Federated Survival Forests: A Comparative Study on Heart Failure and Breast Cancer Genomicscited by 6
  23. 2023Memory-adaptive Depth-wise Heterogeneous Federated Learningcited by 5
  24. 2024FedMedICL: Towards Holistic Evaluation of Distribution Shifts in Federated Medical Imagingcited by 5
  25. 2024A Federated Learning-Friendly Approach for Parameter-Efficient Fine-Tuning of SAM in 3D Segmentationcited by 5
  26. 2025A Thorough Assessment of the Non-IID Data Impact in Federated Learningcited by 5
  27. 2023Heterogeneous Datasets for Federated Survival Analysis Simulationcited by 4
  28. 2023FCA: Taming Long-tailed Federated Medical Image Classification by Classifier Anchoringcited by 4
  29. 2023Federated Model Aggregation via Self-Supervised Priors for Highly Imbalanced Medical Image Classificationcited by 4
  30. 2024A Universal Metric of Dataset Similarity for Cross-silo Federated Learningcited by 4
  31. 2024FedSAC: Dynamic Submodel Allocation for Collaborative Fairness in Federated Learningcited by 4
  32. 2024Federated Learning under Partially Class-Disjoint Data via Manifold Reshapingcited by 4
  33. 2024Probing the Efficacy of Federated Parameter-Efficient Fine-Tuning of Vision Transformers for Medical Image Classificationcited by 4
  34. 2024Enhancing Privacy in Federated Learning: Secure Aggregation for Real-World Healthcare Applicationscited by 4
  35. 2025CYCle: Choosing Your Collaborators Wisely to Enhance Collaborative Fairness in Decentralized Learningcited by 4
  36. 2025Connecting Federated ADMM to Bayescited by 4
  37. 2023SRATTA : Sample Re-ATTribution Attack of Secure Aggregation in Federated Learningcited by 3
  38. 2024SPAM: Stochastic Proximal Point Method with Momentum Variance Reduction for Non-convex Cross-Device Federated Learningcited by 3

50 most cited of 121 All 121 (JSON)