mlsanityData to 14 Oct 2026

The Fundamental Price of Secure Aggregation in Differentially Private Federated Learning

arXiv 2203.03761 ↗First posted 7 Mar 2022cs.LG stat.MLcited by 39

Not on the map. Why?

Abstract (arXiv)

We consider the problem of training a d dimensional model with distributed differential privacy (DP) where secure aggregation (SecAgg) is used to ensure that the server only sees the noisy sum of n model updates in every training round. Taking into account the constraints imposed by SecAgg, we characterize the fundamental communication cost required to obtain the best accuracy achievable under varepsilon central DP (i.e. under a fully trusted server and no communication constraints). Our results show that tildeOleft( min(n2varepsilon2, d) right) bits per client are both sufficient and necessary, and this fundamental limit can be achieved by a linear scheme based on sparse random projections. This provides a significant improvement relative to state-of-the-art SecAgg distributed DP schemes which use tildeO(dlog(d/varepsilon2)) bits per client. Empirically, we evaluate our proposed scheme on real-world federated learning tasks. We find that our theoretical analysis is well matched in practice. In particular, we show that we can reduce the communication cost significantly to under 1.2 bits per parameter in realistic privacy settings without decreasing test-time performance. Our work hence theoretically and empirically specifies the fundamental price of using SecAgg.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Builds on 38 of its 67 references that are in the map. First posted on arXiv 7 Mar 2022.

Builds on

38 of 67 references.

References · 38

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2019Advances and Open Problems in Federated Learningcited by 3,046
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  4. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  5. 2019Towards Federated Learning at Scale: System Designcited by 832
  6. 2018LEAF: A Benchmark for Federated Settingscited by 792
  7. 2017Renyi Differential Privacycited by 650
  8. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 571
  9. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 520
  10. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 478
  11. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 459
  12. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 346
26 more
  1. 2017Sparse Communication for Distributed Gradient Descentcited by 249
  2. 2019Differentially Private Learning with Adaptive Clippingcited by 220
  3. 2017Gradient Sparsification for Communication-Efficient Distributed Optimizationcited by 201
  4. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 179
  5. 2016Practical Secure Aggregation for Federated Learning on User-Held Datacited by 178
  6. 2018Amplification by Shuffling: From Local to Central Differential Privacy via Anonymitycited by 164
  7. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 158
  8. 2016Distributed Mean Estimation with Limited Communicationcited by 136
  9. 2020The Discrete Gaussian for Differential Privacycited by 135
  10. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 128
  11. 2019The Privacy Blanket of the Shuffle Modelcited by 74
  12. 2020Breaking the Communication-Privacy-Accuracy Trilemmacited by 68
  13. 2021The Skellam Mechanism for Differentially Private Federated Learningcited by 68
  14. 2020FastSecAgg: Scalable Secure Aggregation for Privacy-Preserving Federated Learningcited by 61
  15. 2016Discrete Distribution Estimation under Local Privacycited by 59
  16. 2018Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameterscited by 58
  17. 2020Private Summation in the Multi-Message Shuffle Modelcited by 34
  18. 2019Scalable Model Compression by Entropy Penalized Reparameterizationcited by 31
  19. 2020A Primer on Private Statisticscited by 30
  20. 2020A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergencescited by 21
  21. 2020FedSKETCH: Communication-Efficient and Private Federated Learning via Sketchingcited by 20
  22. 2019Separating Local & Shuffled Differential Privacy via Histogramscited by 16
  23. 2021Private Counting from Anonymous Messages: Near-Optimal Accuracy with Vanishing Communication Overheadcited by 16
  24. 2019Private Aggregation from Fewer Anonymous Messagescited by 15
  25. 2020Pure Differentially Private Summation from Anonymous Messagescited by 15
  26. 2020Connecting Robust Shuffle Privacy and Pan-Privacycited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 8; 2023: 13; 2024: 8; 2025: 7; 2026: 3.

39 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
  2. 2023Federated Learning of Gboard Language Models with Differential Privacycited by 66
  3. 2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36
  4. 2024Teach LLMs to Phish: Stealing Private Information from Language Modelscited by 32
  5. 2022On the (In)security of Peer-to-Peer Decentralized Machine Learningcited by 23
  6. 2022Machine Unlearning of Federated Clusterscited by 22
  7. 2023Privacy Amplification via Compression: Achieving the Optimal Privacy-Accuracy-Communication Trade-off in Distributed Mean Estimationcited by 22
  8. 2023Just One Byte (per gradient): A Note on Low-Bandwidth Decentralized Language Model Finetuning Using Shared Randomnesscited by 18
  9. 2023Reconstructing Individual Data Points in Federated Learning Hardened with Differential Privacy and Secure Aggregationcited by 16
  10. 2022Distributed Differential Privacy in Multi-Armed Banditscited by 14
  11. 2024Towards Federated RLHF with Aggregated Client Preference for LLMscited by 13
  12. 2023Compression with Exact Error Distribution for Federated Learningcited by 11
27 more
  1. 2022ScionFL: Efficient and Robust Secure Quantized Aggregationcited by 8
  2. 2024Byzantine-Resilient Secure Aggregation for Federated Learning Without Privacy Compromisescited by 7
  3. 2022Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Designcited by 6
  4. 2023Personalised Federated Learning On Heterogeneous Feature Spacescited by 5
  5. 2023Private Federated Learning with Autotuned Compressioncited by 5
  6. 2023Toward the Tradeoffs between Privacy, Fairness and Utility in Federated Learningcited by 4
  7. 2024Improved Communication-Privacy Trade-offs in L2 Mean Estimation under Streaming Differential Privacycited by 4
  8. 2024Correlated Privacy Mechanisms for Differentially Private Distributed Mean Estimationcited by 4
  9. 2023Breaking the Communication-Privacy-Accuracy Tradeoff with f-Differential Privacycited by 3
  10. 2024Review of Mathematical Optimization in Federated Learningcited by 3
  11. 2022Differentially private partitioned variational inferencecited by 2
  12. 2023Multi-Message Shuffled Privacy in Federated Learningcited by 2
  13. 2025DOME: Improving Signal-to-Noise in Stochastic Gradient Descent via Sharp-Direction Subspace Filteringcited by 2
  14. 2023Optimal Compression of Unit Norm Vectors in the High Distortion Regimecited by 1
  15. 2023Robust and Actively Secure Serverless Collaborative Learningcited by 1
  16. 2024Decomposable Submodular Maximization in Federated Settingcited by 1
  17. 2024On Using Secure Aggregation in Differentially Private Federated Learning with Multiple Local Stepscited by 1
  18. 2025Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Frameworkcited by 1
  19. 2025Perfect Privacy for Discriminator-Based Byzantine-Resilient Federated Learningcited by 1
  20. 2025Decentralized Trust for Space AI: Blockchain-Based Federated Learning Across Multi-Vendor LEO Satellite Networkscited by 1
  21. 2022Private Blind Model Averaging - Distributed, Non-interactive, and Convergentcited by 0
  22. 2025A Selective Homomorphic Encryption Approach for Faster Privacy-Preserving Federated Learningcited by 0
  23. 2025Beyond Trade-offs: A Unified Framework for Privacy, Robustness, and Communication Efficiency in Federated Learningcited by 0
  24. 2025FedRP: A Communication-Efficient Approach for Differentially Private Federated Learning Using Random Projectioncited by 0
  25. 2026Scalable and Private Federated Learning Using Distributed Differential Privacy and Secure Aggregationcited by 0
  26. 2026Local Differential Privacy with Correlated Noise Achieves Central-DP Optimal Costcited by 0
  27. 2026Your Privacy My Cloak: Backdoor Attacks on Differentially Private Federated Learningcited by 0