mlsanityData to 7 Oct 2026

Federated Learning in Non-IID Settings Aided by Differentially Private Synthetic Data

arXiv 2206.00686 ↗First posted 1 Jun 2022cs.LG cs.CR cs.DCcited by 12

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is a privacy-promoting framework that enables potentially large number of clients to collaboratively train machine learning models. In a FL system, a server coordinates the collaboration by collecting and aggregating clients' model updates while the clients' data remains local and private. A major challenge in federated learning arises when the local data is heterogeneous -- the setting in which performance of the learned global model may deteriorate significantly compared to the scenario where the data is identically distributed across the clients. In this paper we propose FedDPMS (Federated Differentially Private Means Sharing), an FL algorithm in which clients deploy variational auto-encoders to augment local datasets with data synthesized using differentially private means of latent data representations communicated by a trusted server. Such augmentation ameliorates effects of data heterogeneity across the clients without compromising privacy. Our experiments on deep image classification tasks demonstrate that FedDPMS outperforms competing state-of-the-art FL methods specifically designed for heterogeneous data settings.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 46 of its 64 references that are in the map. First posted on arXiv 1 Jun 2022.

Builds on

46 of 64 references.

References · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  7. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  8. 2019Advances and Open Problems in Federated Learningcited by 3,021
  9. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  10. 2016Deep Learning with Differential Privacycited by 2,682
  11. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  12. 2020Adaptive Federated Optimizationcited by 1,036
34 more
  1. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  2. 2021Model-Contrastive Federated Learningcited by 762
  3. 2017Renyi Differential Privacycited by 643
  4. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  5. 2020Federated Learning with Matched Averagingcited by 564
  6. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  7. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  8. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  9. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  10. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  11. 2019Federated Machine Learning: Concept and Applicationscited by 409
  12. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  13. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 367
  14. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 326
  15. 2021FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Spacecited by 284
  16. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 218
  17. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  18. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 174
  19. 2020DP-CGAN: Differentially Private Synthetic Data and Label Generationcited by 118
  20. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 112
  21. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  22. 2020Provable Defense against Privacy Leakage in Federated Learning from Representation Perspectivecited by 94
  23. 2021Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learningcited by 93
  24. 2020Federated Unsupervised Representation Learningcited by 92
  25. 2020FedGAN: Federated Generative Adversarial Networks for Distributed Datacited by 89
  26. 2019Generative Models for Effective ML on Private, Decentralized Datasetscited by 85
  27. 2021Towards Fair Federated Learning with Zero-Shot Data Augmentationcited by 64
  28. 2019Practical Federated Gradient Boosting Decision Treescited by 53
  29. 2021PipeTransformer: Automated Elastic Pipelining for Distributed Training of Transformerscited by 23
  30. 2019Federated Generative Privacycited by 16
  31. 2020Federating Recommendations Using Differentially Private Prototypescited by 13
  32. 2021Federated Variational Learning for Anomaly Detection in Multivariate Time Seriescited by 12
  33. 2016On-Average KL-Privacy and its equivalence to Generalization for Max-Entropy Mechanismscited by 10
  34. 2021Privacy-Preserving Self-Taught Federated Learning for Heterogeneous Datacited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 2; 2026: 2.

12 papers.

  1. 2023The Best of Both Worlds: Accurate Global and Personalized Models through Federated Learning with Data-Free Hyper-Knowledge Distillationcited by 25
  2. 2023Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devicescited by 11
  3. 2024Recovering Labels from Local Updates in Federated Learningcited by 5
  4. 2024A Systematic Review of Federated Generative Modelscited by 5
  5. 2024FedCFA: Alleviating Simpson's Paradox in Model Aggregation with Counterfactual Federated Learningcited by 4
  6. 2023Heterogeneity-Guided Client Sampling: Towards Fast and Efficient Non-IID Federated Learningcited by 2
  7. 2025Enhancing Visual Representation with Textual Semantics: Textual Semantics-Powered Prototypes for Heterogeneous Federated Learningcited by 2
  8. 2024Federated Learning for Traffic Flow Prediction with Synthetic Data Augmentationcited by 1
  9. 2024FissionVAE: Federated Non-IID Image Generation with Latent Space and Decoder Decompositioncited by 0
  10. 2025Differentially Private Decentralized Dataset Synthesis Through Randomized Mixing with Correlated Noisecited by 0
  11. 2026GFPL: Generative Federated Prototype Learning for Resource-Constrained and Data-Imbalanced Vision Taskcited by 0
  12. 2026Federated Deep Clustering Networks for High-Dimensional and Heterogeneous Datacited by 0