FedAUXfdp: Differentially Private One-Shot Federated Distillation
Not on the map. Why?
Abstract (arXiv)
Federated learning suffers in the case of non-iid local datasets, i.e., when the distributions of the clients' data are heterogeneous. One promising approach to this challenge is the recently proposed method FedAUX, an augmentation of federated distillation with robust results on even highly heterogeneous client data. FedAUX is a partially (ε, δ)-differentially private method, insofar as the clients' private data is protected in only part of the training it takes part in. This work contributes a fully differentially private modification, termed FedAUXfdp. We further contribute an upper bound on the l2-sensitivity of regularized multinomial logistic regression. In experiments with deep networks on large-scale image datasets, FedAUXfdp with strong differential privacy guarantees performs significantly better than other equally privatized SOTA baselines on non-iid client data in just a single communication round. Full privatization of the modified method results in a negligible reduction in accuracy at all levels of data heterogeneity.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 22 of its 38 references that are in the map. First posted on arXiv 30 May 2022.
Builds on
22 of 38 references.
References · 22
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
- 2019Advances and Open Problems in Federated Learningcited by 3,046
- 2016Deep Learning with Differential Privacycited by 2,728
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
- 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
- 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
- 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
- 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
- 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 459
10 more
- 2020Label-Only Membership Inference Attackscited by 388
- 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 374
- 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 348
- 2018Dataset Distillationcited by 230
- 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
- 2020Distilled One-Shot Federated Learningcited by 109
- 2019Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfercited by 93
- 2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 69
- 2020Federated Model Distillation with Noise-Free Differential Privacycited by 40
- 2021FedH2L: Federated Learning with Model and Statistical Heterogeneitycited by 17
Built on by
Citing papers per year: 2023: 1; 2024: 2.
3 papers.