Data-Free Black-Box Federated Learning via Zeroth-Order Gradient Estimation
Not on the map. Why?
Abstract (arXiv)
Federated learning (FL) enables decentralized clients to collaboratively train a global model under the orchestration of a central server without exposing their individual data. However, the iterative exchange of model parameters between the server and clients imposes heavy communication burdens, risks potential privacy leakage, and even precludes collaboration among heterogeneous clients. Distillation-based FL tackles these challenges by exchanging low-dimensional model outputs rather than model parameters, yet it highly relies on a task-relevant auxiliary dataset that is often not available in practice. Data-free FL attempts to overcome this limitation by training a server-side generator to directly synthesize task-specific data samples for knowledge transfer. However, the update rule of the generator requires clients to share on-device models for white-box access, which greatly compromises the advantages of distillation-based FL. This motivates us to explore a data-free and black-box FL framework via Zeroth-order Gradient Estimation (FedZGE), which estimates the gradients after flowing through on-device models in a black-box optimization manner to complete the training of the generator in terms of fidelity, transferability, diversity, and equilibrium, without involving any auxiliary data or sharing any model parameters, thus combining the advantages of both distillation-based FL and data-free FL. Experiments on large-scale image classification datasets and network architectures demonstrate the superiority of FedZGE in terms of data heterogeneity, model heterogeneity, communication efficiency, and privacy protection.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 43 references that are in the map. First posted on arXiv 8 Mar 2025.
Builds on
24 of 43 references.
References · 24
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2014Conditional Generative Adversarial Netscited by 2,162
- 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
- 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
- 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
- 2019Data-Free Learning of Student Networkscited by 204
- 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
- 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 142
- 2019Mode Seeking Generative Adversarial Networks for Diverse Image Synthesiscited by 138
- 2017Data-Free Knowledge Distillation for Deep Neural Networkscited by 136
12 more
- 2019Zero-shot Knowledge Transfer via Adversarial Belief Matchingcited by 129
- 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
- 2018Zeroth-Order Stochastic Variance Reduction for Nonconvex Optimizationcited by 111
- 2019Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfercited by 93
- 2019Data-Free Adversarial Distillationcited by 90
- 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 55
- 2021FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusioncited by 49
- 2022Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillationcited by 45
- 2022A Survey on Heterogeneous Federated Learningcited by 42
- 2021FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Modelscited by 31
- 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 24
- 2023Knowledge Distillation in Federated Edge Learning: A Surveycited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.