mlsanityData to 7 Oct 2026

Federated Learning from Small Datasets

arXiv 2110.03469 ↗First posted 7 Oct 2021cs.LG cs.AI cs.DCcited by 14

Not on the map. Why?

Abstract (arXiv)

Federated learning allows multiple parties to collaboratively train a joint model without sharing local data. This enables applications of machine learning in settings of inherently distributed, undisclosable data such as in the medical domain. In practice, joint training is usually achieved by aggregating local models, for which local training objectives have to be in expectation similar to the joint (global) objective. Often, however, local datasets are so small that local objectives differ greatly from the global objective, resulting in federated learning to fail. We propose a novel approach that intertwines model aggregations with permutations of local models. The permutations expose each local model to a daisy chain of local datasets resulting in more efficient training in data-sparse domains. This enables training on extremely small local datasets, such as patient data across hospitals, while retaining the training efficiency and privacy benefits of federated learning.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 30 of its 62 references that are in the map. First posted on arXiv 7 Oct 2021.

Builds on

30 of 62 references.

References · 30

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  3. 2016Matching Networks for One Shot Learningcited by 2,246
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  5. 2020Adaptive Federated Optimizationcited by 1,036
  6. 2019Deep Leakage from Gradientscited by 1,004
  7. 2021Model-Contrastive Federated Learningcited by 762
  8. 2020The Future of Digital Health with Federated Learningcited by 625
  9. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  10. 2020Federated Learning with Matched Averagingcited by 564
  11. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  12. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
18 more
  1. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  2. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  3. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  4. 2018Analyzing Federated Learning through an Adversarial Lenscited by 346
  5. 2019Robust Aggregation for Federated Learningcited by 314
  6. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
  7. 2018Few-shot Object Detection via Feature Reweightingcited by 265
  8. 2019Can You Really Backdoor Federated Learning?cited by 250
  9. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
  10. 2019On the Convergence of Local Descent Methods in Federated Learningcited by 150
  11. 2013Hacking Smart Machines with Smarter Ones: How to Extract Meaningful Data from Machine Learning Classifierscited by 128
  12. 2020Robust Federated Learning: The Case of Affine Distribution Shiftscited by 102
  13. 2021Personalized Federated Learning with Gaussian Processescited by 76
  14. 2021Towards Fair Federated Learning with Zero-Shot Data Augmentationcited by 64
  15. 2019On Safeguarding Privacy and Security in the Framework of Federated Learningcited by 55
  16. 2021FLOP: Federated Learning on Medical Datasets using Partial Networkscited by 42
  17. 2018Efficient Decentralized Deep Learning by Dynamic Model Averagingcited by 30
  18. 2018Effective Parallelisation for Machine Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 2; 2025: 2; 2026: 3.

14 papers.

Sort:

  1. 2022Federated Learning and Meta Learning: Approaches, Applications, and Directionscited by 12
  2. 2025Probabilistic Federated Prompt-Tuning with Non-IID and Imbalanced Datacited by 9
  3. 2024One-Shot Sequential Federated Learning for Non-IID Data by Enhancing Local Model Diversitycited by 8
  4. 2023A Privacy Preserving System for Movie Recommendations Using Federated Learningcited by 6
  5. 2023SkyMask: Attack-agnostic Robust Federated Learning with Fine-grained Learnable Maskscited by 4
  6. 2023FedDBL: Communication and Data Efficient Federated Deep-Broad Learning for Histopathological Tissue Classificationcited by 3
  7. 2023Keep It Simple: Fault Tolerance Evaluation of Federated Learning with Unreliable Clientscited by 2
  8. 2023PPFL: A Personalized Federated Learning Framework for Heterogeneous Populationcited by 1
  9. 2024Provable Mutual Benefits from Federated Learning in Privacy-Sensitive Domainscited by 1
  10. 2023Factor-Assisted Federated Learning for Personalized Optimization with Heterogeneous Datacited by 0
  11. 2025SFedKD: Sequential Federated Learning with Discrepancy-Aware Multi-Teacher Knowledge Distillationcited by 0
  12. 2026Safeguarding Privacy: Privacy-Preserving Detection of Mind Wandering and Disengagement Using Federated Learning in Online Educationcited by 0
2 more
  1. 2026When Generator Replay Degrades: Projected Rehearsal Orchestration for Heterogeneous Federated Class-Incremental Learningcited by 0
  2. 2026A Trust-Network-Based Federated Learning Framework for Multi-Center Aging Clock Predictioncited by 0