mlsanityData to 7 Oct 2026

DFDG: Data-Free Dual-Generator Adversarial Distillation for One-Shot Federated Learning

arXiv 2409.07734 ↗First posted 12 Sep 2024cs.DC cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a distributed machine learning scheme in which clients jointly participate in the collaborative training of a global model by sharing model information rather than their private datasets. In light of concerns associated with communication and privacy, one-shot FL with a single communication round has emerged as a de facto promising solution. However, existing one-shot FL methods either require public datasets, focus on model homogeneous settings, or distill limited knowledge from local models, making it difficult or even impractical to train a robust global model. To address these limitations, we propose a new data-free dual-generator adversarial distillation method (namely DFDG) for one-shot FL, which can explore a broader local models' training space via training dual generators. DFDG is executed in an adversarial manner and comprises two parts: dual-generator training and dual-model distillation. In dual-generator training, we delve into each generator concerning fidelity, transferability and diversity to ensure its utility, and additionally tailor the cross-divergence loss to lessen the overlap of dual generators' output spaces. In dual-model distillation, the trained dual generators work together to provide the training data for updates of the global model. At last, our extensive experiments on various image classification tasks show that DFDG achieves significant performance gains in accuracy compared to SOTA baselines.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 43 of its 59 references that are in the map. First posted on arXiv 12 Sep 2024.

Builds on

43 of 59 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  6. 2021Model-Contrastive Federated Learningcited by 762
  7. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  8. 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 562
  9. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  10. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  11. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  12. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
31 more
  1. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  2. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  3. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  4. 2021Federated Learning for Internet of Things: A Comprehensive Surveycited by 247
  5. 2019Data-Free Learning of Student Networkscited by 203
  6. 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
  7. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  8. 2019One-Shot Federated Learningcited by 124
  9. 2019Zero-Shot Knowledge Distillation in Deep Networkscited by 119
  10. 2020Distilled One-Shot Federated Learningcited by 108
  11. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  12. 2021DENSE: Data-Free One-Shot Federated Learningcited by 100
  13. 2021Heterogeneity for the Win: One-Shot Federated Clusteringcited by 87
  14. 2020Practical One-Shot Federated Learning for Cross-Silo Settingcited by 76
  15. 2021Contrastive Model Inversion for Data-Free Knowledge Distillationcited by 66
  16. 2021HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Imagescited by 62
  17. 2020XOR Mixup: Privacy-Preserving Data Augmentation for One-Shot Federated Learningcited by 58
  18. 2022Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillationcited by 45
  19. 2022Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customizationcited by 34
  20. 2022Multi-Level Branched Regularization for Federated Learningcited by 34
  21. 2022Momentum Adversarial Distillation: Handling Large Distribution Shifts in Data-Free Knowledge Distillationcited by 31
  22. 2022Learning to Invert: Simple Adaptive Attacks for Gradient Inversion in Federated Learningcited by 30
  23. 2020Large-Scale Generative Data-Free Distillationcited by 29
  24. 2023GradMA: A Gradient-Memory-based Accelerated Federated Learning with Alleviated Catastrophic Forgettingcited by 27
  25. 2023One-Shot Federated Conformal Predictioncited by 26
  26. 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 23
  27. 2019Man-in-the-Middle Attacks against Machine Learning Classifiers via Malicious Generative Modelscited by 19
  28. 2023Dual Student Networks for Data-Free Model Stealingcited by 17
  29. 2021Data-Free Knowledge Distillation with Soft Targeted Transfer Set Synthesiscited by 14
  30. 2023Decentralized Local Updates with Dual-Slow Estimation and Momentum-based Variance-Reduction for Non-Convex Optimizationcited by 2
  31. 2024Privacy-Preserving Federated Learning with Consistency via Knowledge Distillation Using Conditional Generatorcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Towards One-shot Federated Learning: Advances, Challenges, and Future Directionscited by 2
  2. 2025Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environmentscited by 2