mlsanityData to 7 Oct 2026

Reconciling Security and Communication Efficiency in Federated Learning

arXiv 2207.12779 ↗First posted 26 Jul 2022cs.LG cs.AI cs.DCcited by 4

Not on the map. Why?

Abstract (arXiv)

Cross-device Federated Learning is an increasingly popular machine learning setting to train a model by leveraging a large population of client devices with high privacy and security guarantees. However, communication efficiency remains a major bottleneck when scaling federated learning to production environments, particularly due to bandwidth constraints during uplink communication. In this paper, we formalize and address the problem of compressing client-to-server model updates under the Secure Aggregation primitive, a core component of Federated Learning pipelines that allows the server to aggregate the client updates without accessing them individually. In particular, we adapt standard scalar quantization and pruning methods to Secure Aggregation and propose Secure Indexing, a variant of Secure Aggregation that supports quantization for extreme compression. We establish state-of-the-art results on LEAF benchmarks in a secure Federated Learning setup with up to 40× compression in uplink communication with no meaningful loss in utility compared to uncompressed baselines.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 48 of its 70 references that are in the map. First posted on arXiv 26 Jul 2022.

Builds on

48 of 70 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2020Extracting Training Data from Large Language Modelscited by 2,144
  7. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  8. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  9. 2021Membership Inference Attacks From First Principlescited by 839
  10. 2018LEAF: A Benchmark for Federated Settingscited by 780
  11. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  12. 2020What is the State of Neural Network Pruning?cited by 578
36 more
  1. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  2. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  3. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  4. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
  5. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  6. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 471
  7. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  8. 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 391
  9. 2015Deep Learning with Limited Numerical Precisioncited by 368
  10. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 253
  11. 2017Sparse Communication for Distributed Gradient Descentcited by 243
  12. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 234
  13. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 219
  14. 2017Gradient Sparsification for Communication-Efficient Distributed Optimizationcited by 199
  15. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
  16. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 178
  17. 2021On the Importance of Difficulty Calibration in Membership Inference Attackscited by 137
  18. 2018ATOMO: Communication-efficient Learning via Atomic Sparsificationcited by 135
  19. 2019DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compressioncited by 98
  20. 2021Papaya: Practical, Private, and Scalable Federated Learningcited by 89
  21. 2020Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approachcited by 80
  22. 2019And the Bit Goes Down: Revisiting the Quantization of Neural Networkscited by 78
  23. 2021On Large-Cohort Training for Federated Learningcited by 73
  24. 2020Federated Learning With Quantized Global Model Updatescited by 54
  25. 2020Can Federated Learning Save The Planet?cited by 52
  26. 2019Federated Learning Of Out-Of-Vocabulary Wordscited by 51
  27. 2019Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedbackcited by 49
  28. 2018SparCML: High-Performance Sparse Communication for Machine Learningcited by 34
  29. 2017AdaComp : Adaptive Residual Gradient Compression for Data-Parallel Distributed Trainingcited by 33
  30. 2019A Double Residual Compression Algorithm for Efficient Distributed Learningcited by 31
  31. 2019Federated Learning with Autotuned Communication-Efficient Secure Aggregationcited by 23
  32. 2018GradiVeQ: Vector Quantization for Bandwidth-Efficient Gradient Aggregation in Distributed CNN Trainingcited by 21
  33. 2021Preserved central model for faster bidirectional compression in distributed settingscited by 21
  34. 2022Privacy-Aware Compression for Federated Data Analysiscited by 17
  35. 2022FedLite: A Scalable Approach for Federated Learning on Resource-constrained Clientscited by 14
  36. 2021DP-REC: Private & Communication-Efficient Federated Learningcited by 10

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 1; 2024: 1.

4 papers.

  1. 2022Distributed Contrastive Learning for Medical Image Segmentationcited by 16
  2. 2023Green Federated Learningcited by 8
  3. 2022PersA-FL: Personalized Asynchronous Federated Learningcited by 5
  4. 2024FedMPQ: Secure and Communication-Efficient Federated Learning with Multi-codebook Product Quantizationcited by 1