mlsanityData to 14 Oct 2026

DENSE: Data-Free One-Shot Federated Learning

arXiv 2112.12371 ↗First posted 23 Dec 2021cs.LG cs.CVcited by 100

Not on the map. Why?

Abstract (arXiv)

One-shot Federated Learning (FL) has recently emerged as a promising approach, which allows the central server to learn a model in a single communication round. Despite the low communication cost, existing one-shot FL methods are mostly impractical or face inherent limitations, eg a public dataset is required, clients' models are homogeneous, and additional data/model information need to be uploaded. To overcome these issues, we propose a novel two-stage Data-freE oNe-Shot federated lEarning (DENSE) framework, which trains the global model by a data generation stage and a model distillation stage. DENSE is a practical one-shot FL method that can be applied in reality due to the following advantages: (1) DENSE requires no additional information compared with other methods (except the model parameters) to be transferred between clients and the server; (2) DENSE does not require any auxiliary dataset for training; (3) DENSE considers model heterogeneity in FL, ie different clients can have different model architectures. Experiments on a variety of real-world datasets demonstrate the superiority of our method.For example, DENSE outperforms the best baseline method Fed-ADI by 5.08% on CIFAR10 dataset.

Key facts. Cited by 100 papers in the map (data through 14 Oct 2026). Builds on 37 of its 63 references that are in the map. First posted on arXiv 23 Dec 2021.

Builds on

37 of 63 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  6. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  7. 2018Federated Learning with Non-IID Datacited by 1,108
  8. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 989
  9. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  10. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  11. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  12. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
25 more
  1. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 502
  2. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  3. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  4. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  5. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 334
  6. 2021See through Gradients: Image Batch Recovery via GradInversioncited by 326
  7. 2021FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Spacecited by 287
  8. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  9. 2020Privacy and Robustness in Federated Learning: Attacks and Defensescited by 166
  10. 2020M2m: Imbalanced Classification via Major-to-minor Translationcited by 134
  11. 2019One-Shot Federated Learningcited by 125
  12. 2020Distilled One-Shot Federated Learningcited by 109
  13. 2021Heterogeneity for the Win: One-Shot Federated Clusteringcited by 87
  14. 2020Practical One-Shot Federated Learning for Cross-Silo Settingcited by 77
  15. 2021Pufferfish: Communication-efficient Models At No Extra Costcited by 61
  16. 2021CMUA-Watermark: A Cross-Model Universal Adversarial Watermark for Combating Deepfakescited by 56
  17. 2020What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentationcited by 50
  18. 2018Knowledge Distillation with Adversarial Samples Supporting Decision Boundarycited by 49
  19. 2020Federated Model Distillation with Noise-Free Differential Privacycited by 40
  20. 2019PPGAN: Privacy-preserving Generative Adversarial Networkcited by 34
  21. 2020FLAME: Differentially Private Federated Learning in the Shuffle Modelcited by 27
  22. 2021Personalized Retrogress-Resilient Framework for Real-World Medical Federated Learningcited by 25
  23. 2019Man-in-the-Middle Attacks against Machine Learning Classifiers via Malicious Generative Modelscited by 19
  24. 2020Incremental Embedding Learning via Zero-Shot Translationcited by 11
  25. 2021Private Hierarchical Clustering in Federated Networkscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 24; 2024: 40; 2025: 22; 2026: 9.

100 papers.

Sort:

By area:

  1. 2023Deep Model Fusion: A Surveycited by 91
  2. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 84
  3. 2023TARGET: Federated Class-Continual Learning via Exemplar-Free Distillationcited by 65
  4. 2023No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifiercited by 55
  5. 2023Federated Incremental Semantic Segmentationcited by 36
  6. 2024Evaluations of Machine Learning Privacy Defenses are Misleadingcited by 36
  7. 2022TranSpeech: Speech-to-Speech Translation With Bilateral Perturbationcited by 34
  8. 2023Exploring One-shot Semi-supervised Federated Learning with A Pre-trained Diffusion Modelcited by 33
  9. 2023Dataset Distillation via the Wasserstein Metriccited by 31
  10. 2024Enhancing One-Shot Federated Learning Through Data and Ensemble Co-Boostingcited by 27
  11. 2024Federated Large Language Models: Current Progress and Future Directionscited by 27
  12. 2025The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutionscited by 26
38 more
  1. 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 24
  2. 2023Federated Generative Learning with Foundation Modelscited by 23
  3. 2023FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregationcited by 22
  4. 2024Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchorscited by 21
  5. 2022One-shot Federated Learning without Server-side Trainingcited by 20
  6. 2024DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillationcited by 20
  7. 2023Delving into the Adversarial Robustness of Federated Learningcited by 19
  8. 2023Learning to Learn from APIs: Black-Box Data-Free Meta-Learningcited by 18
  9. 2024FedDEO: Description-Enhanced One-Shot Federated Learning with Diffusion Modelscited by 17
  10. 2022IDEAL: Query-Efficient Data-Free Learning from Black-box Modelscited by 16
  11. 2024Text-Enhanced Data-free Approach for Federated Class-Incremental Learningcited by 16
  12. 2024FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Modelscited by 16
  13. 2024FedFisher: Leveraging Fisher Information for One-Shot Federated Learningcited by 15
  14. 2024Federated Distillation: A Surveycited by 15
  15. 2023Byzantine-Robust Learning on Heterogeneous Data via Gradient Splittingcited by 14
  16. 2024Revisiting Ensembling in One-Shot Federated Learningcited by 13
  17. 2024Task Arithmetic Through The Lens Of One-Shot Federated Learningcited by 12
  18. 2024One-shot Federated Learning via Synthetic Distiller-Distillate Communicationcited by 12
  19. 2024FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusioncited by 11
  20. 2023Revisiting Data-Free Knowledge Distillation with Poisoned Teacherscited by 9
  21. 2023MAS: Towards Resource-Efficient Federated Multiple-Task Learningcited by 9
  22. 2024Applications of Generative AI (GAI) for Mobile and Wireless Networking: A Surveycited by 9
  23. 2024Knowledge Distillation in Federated Learning: a Survey on Long Lasting Challenges and New Solutionscited by 9
  24. 2023Sampling to Distill: Knowledge Transfer from Open-World Datacited by 8
  25. 2024One-Shot Sequential Federated Learning for Non-IID Data by Enhancing Local Model Diversitycited by 8
  26. 2023Generalized Universal Domain Adaptation with Generative Flow Networkscited by 7
  27. 2024Navigating Heterogeneity and Privacy in One-Shot Federated Learning with Diffusion Modelscited by 7
  28. 2025One-Shot Federated Learning with Classifier-Free Diffusion Modelscited by 7
  29. 2023Fake It Till Make It: Federated Learning with Consensus-Oriented Generationcited by 6
  30. 2025One-shot Federated Learning Methods: A Practical Guidecited by 6
  31. 2025Capture Global Feature Statistics for One-Shot Federated Learningcited by 6
  32. 2025Federated Learning for Medical Image Classification: A Comprehensive Benchmarkcited by 6
  33. 2023One-Shot Heterogeneous Federated Learning with Local Model-Guided Diffusion Modelscited by 5
  34. 2024A Systematic Review of Federated Generative Modelscited by 5
  35. 2024Local Superior Soups: A Catalyst for Model Merging in Cross-Silo Federated Learningcited by 5
  36. 2022Boosting Pruned Networks with Linear Over-parameterizationcited by 4
  37. 2023Task-Distributionally Robust Data-Free Meta-Learningcited by 4
  38. 2024DFML: Decentralized Federated Mutual Learningcited by 4

50 most cited of 100 All 100 (JSON)