mlsanityData to 14 Oct 2026

Mitigating Data Heterogeneity in Federated Learning with Data Augmentation

arXiv 2206.09979 ↗First posted 20 Jun 2022cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a prominent framework that enables training a centralized model while securing user privacy by fusing local, decentralized models. In this setting, one major obstacle is data heterogeneity, i.e., each client having non-identically and independently distributed (non-IID) data. This is analogous to the context of Domain Generalization (DG), where each client can be treated as a different domain. However, while many approaches in DG tackle data heterogeneity from the algorithmic perspective, recent evidence suggests that data augmentation can induce equal or greater performance. Motivated by this connection, we present federated versions of popular DG algorithms, and show that by applying appropriate data augmentation, we can mitigate data heterogeneity in the federated setting, and obtain higher accuracy on unseen clients. Equipped with data augmentation, we can achieve state-of-the-art performance using even the most basic Federated Averaging algorithm, with much sparser communication.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 46 of its 69 references that are in the map. First posted on arXiv 20 Jun 2022.

Builds on

46 of 69 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  8. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  9. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  10. 2019Invariant Risk Minimizationcited by 1,854
  11. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  12. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
34 more
  1. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  2. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,188
  3. 2018Federated Learning with Non-IID Datacited by 1,108
  4. 2020In Search of Lost Domain Generalizationcited by 1,054
  5. 2017Deeper, Broader and Artier Domain Generalizationcited by 986
  6. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  7. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  8. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
  9. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 742
  10. 2013Domain Generalization via Invariant Feature Representationcited by 623
  11. 2020Federated Learning with Matched Averagingcited by 564
  12. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 536
  13. 2019Agnostic Federated Learningcited by 456
  14. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  15. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 312
  16. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 305
  17. 2020Learning to Learn Single Domain Generalizationcited by 296
  18. 2017Data Augmentation Generative Adversarial Networkscited by 278
  19. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  20. 2015Domain Generalization for Object Recognition with Multi-task Autoencoderscited by 253
  21. 2020Representation Learning via Invariant Causal Mechanismscited by 199
  22. 2017Domain Generalization by Marginal Transfer Learningcited by 180
  23. 2019Federated Adversarial Domain Adaptationcited by 140
  24. 2020FedSplit: An algorithmic framework for fast federated optimizationcited by 108
  25. 2017Learning to Compose Domain-Specific Transformations for Data Augmentationcited by 91
  26. 2020A causal framework for distribution generalizationcited by 68
  27. 2022Out-of-distribution Generalization with Causal Invariant Transformationscited by 45
  28. 2021Federated Multi-Target Domain Adaptationcited by 44
  29. 2021Fed2: Feature-Aligned Federated Learningcited by 42
  30. 2018Multi-component Image Translation for Deep Domain Generalizationcited by 40
  31. 2020Federated Transfer Learning: concept and applicationscited by 32
  32. 2016RenderGAN: Generating Realistic Labeled Datacited by 20
  33. 2017Group invariance principles for causal generative modelscited by 20
  34. 2021Towards Causal Federated Learning For Enhanced Robustness and Privacycited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 12; 2024: 2; 2025: 5; 2026: 1.

21 papers.

Sort:

By area:

  1. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 84
  2. 2023Federated Domain Generalization: A Surveycited by 25
  3. 2023Federated Generative Learning with Foundation Modelscited by 23
  4. 2023A Review of the Role of Causality in Developing Trustworthy AI Systemscited by 14
  5. 2023Multimodal Federated Learning in Healthcare: a Reviewcited by 13
  6. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 13
  7. 2023Towards Achieving Near-optimal Utility for Privacy-Preserving Federated Learning via Data Generation and Parameter Distortioncited by 6
  8. 2023Federated Learning under Covariate Shifts with Generalization Guaranteescited by 6
  9. 2025Federated Deconfounding and Debiasing Learning for Out-of-Distribution Generalizationcited by 4
  10. 2023Federated attention consistent learning models for prostate cancer diagnosis and Gleason gradingcited by 2
  11. 2023Topology Learning for Heterogeneous Decentralized Federated Learning over Unreliable D2D Networkscited by 2
  12. 2025FedBM: Stealing Knowledge from Pre-trained Language Models for Heterogeneous Federated Learningcited by 2
9 more
  1. 2023FLAG: Fast Label-Adaptive Aggregation for Multi-label Classification in Federated Learningcited by 1
  2. 2025Addressing Label Shift in Distributed Learning via Entropy Regularizationcited by 1
  3. 2025FedPhD: Federated Pruning with Hierarchical Learning of Diffusion Modelscited by 1
  4. 2022FedVeca: Federated Vectorized Averaging on Non-IID Data with Adaptive Bi-directional Global Objectivecited by 0
  5. 2023Global Layers: Non-IID Tabular Federated Learningcited by 0
  6. 2023Heterogeneous Federated Learning via Personalized Generative Networkscited by 0
  7. 2024Accelerating Federated Learning by Selecting Beneficial Herd of Local Gradientscited by 0
  8. 2025Topology-Aware Knowledge Propagation in Decentralized Learningcited by 0
  9. 2026FedASAP: Activation Statistics-driven Structured Adaptive Pruning for Efficient Personalized Federated Learning for Lesion Segmentation on brain MRIcited by 0