mlsanityData to 7 Oct 2026

Constrained Differentially Private Federated Learning for Low-bandwidth Devices

arXiv 2103.00342 ↗First posted 27 Feb 2021cs.CR cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Federated learning becomes a prominent approach when different entities want to learn collaboratively a common model without sharing their training data. However, Federated learning has two main drawbacks. First, it is quite bandwidth inefficient as it involves a lot of message exchanges between the aggregating server and the participating entities. This bandwidth and corresponding processing costs could be prohibitive if the participating entities are, for example, mobile devices. Furthermore, although federated learning improves privacy by not sharing data, recent attacks have shown that it still leaks information about the training data. This paper presents a novel privacy-preserving federated learning scheme. The proposed scheme provides theoretical privacy guarantees, as it is based on Differential Privacy. Furthermore, it optimizes the model accuracy by constraining the model learning phase on few selected weights. Finally, as shown experimentally, it reduces the upstream and downstream bandwidth by up to 99.9% compared to standard federated learning, making it practical for mobile systems.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 27 of its 54 references that are in the map. First posted on arXiv 27 Feb 2021.

Builds on

27 of 54 references.

References · 27

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2016Deep Learning with Differential Privacycited by 2,682
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  6. 2019Deep Leakage from Gradientscited by 1,004
  7. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  8. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  9. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 471
  10. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  11. 2020iDLG: Improved Deep Leakage from Gradientscited by 367
  12. 2018Scalable and accurate deep learning for electronic health recordscited by 275
15 more
  1. 2014RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Responsecited by 262
  2. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 253
  3. 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 199
  4. 2016Practical Secure Aggregation for Federated Learning on User-Held Datacited by 178
  5. 2019Federated Learning over Wireless Fading Channelscited by 178
  6. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 178
  7. 2018ATOMO: Communication-efficient Learning via Atomic Sparsificationcited by 135
  8. 2016Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1cited by 102
  9. 2019Communication-efficient distributed SGD with Sketchingcited by 81
  10. 2020LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasetscited by 60
  11. 2020FedSel: Federated SGD under Local Differential Privacy with Top-k Dimension Selectioncited by 35
  12. 2016Survey of resampling techniques for improving classification performance in unbalanced datasetscited by 32
  13. 2017Improving Palliative Care with Deep Learningcited by 28
  14. 2020Voting-based Approaches For Differentially Private Federated Learningcited by 14
  15. 2020Compression Boosts Differentially Private Federated Learningcited by 13

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 3; 2024: 1.

6 papers.

  1. 2023Privacy-Aware Document Visual Question Answeringcited by 17
  2. 2022Practical Challenges in Differentially-Private Federated Survival Analysis of Medical Datacited by 7
  3. 2023FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximationscited by 4
  4. 2024NeurIPS 2023 Competition: Privacy Preserving Federated Learning Document VQAcited by 2
  5. 2021The More, the Better? A Study on Collaborative Machine Learning for DGA Detectioncited by 1
  6. 2023ALI-DPFL: Differentially Private Federated Learning with Adaptive Local Iterationscited by 1