mlsanityData to 7 Oct 2026

SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compression

arXiv 2206.09888 ↗First posted 20 Jun 2022cs.LG cs.DC cs.DScited by 34

Not on the map. Why?

Abstract (arXiv)

To enable large-scale machine learning in bandwidth-hungry environments such as wireless networks, significant progress has been made recently in designing communication-efficient federated learning algorithms with the aid of communication compression. On the other end, privacy-preserving, especially at the client level, is another important desideratum that has not been addressed simultaneously in the presence of advanced communication compression techniques yet. In this paper, we propose a unified framework that enhances the communication efficiency of private federated learning with communication compression. Exploiting both general compression operators and local differential privacy, we first examine a simple algorithm that applies compression directly to differentially-private stochastic gradient descent, and identify its limitations. We then propose a unified framework SoteriaFL for private federated learning, which accommodates a general family of local gradient estimators including popular stochastic variance-reduced gradient methods and the state-of-the-art shifted compression scheme. We provide a comprehensive characterization of its performance trade-offs in terms of privacy, utility, and communication complexity, where SoteraFL is shown to achieve better communication complexity without sacrificing privacy nor utility than other private federated learning algorithms without communication compression.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 56 of its 89 references that are in the map. First posted on arXiv 20 Jun 2022.

Builds on

56 of 89 references.

References · 56

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  7. 2019Deep Leakage from Gradientscited by 1,004
  8. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  9. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
  10. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  11. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
  12. 2021Federated Learning Based on Dynamic Regularizationcited by 555
38 more
  1. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  2. 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 401
  3. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  4. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  5. 2021A Field Guide to Federated Optimizationcited by 312
  6. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 306
  7. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  8. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 157
  9. 2020Private Stochastic Convex Optimization: Optimal Rates in Linear Timecited by 152
  10. 2018ATOMO: Communication-efficient Learning via Atomic Sparsificationcited by 135
  11. 2016Distributed Mean Estimation with Limited Communicationcited by 134
  12. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  13. 2021EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedbackcited by 131
  14. 2018Differentially Private Empirical Risk Minimization Revisited: Faster and More Generalcited by 127
  15. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  16. 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 122
  17. 2020PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimizationcited by 116
  18. 2018Distributed Differential Privacy via Shufflingcited by 115
  19. 2020FedSplit: An algorithmic framework for fast federated optimizationcited by 108
  20. 2020Acceleration for Compressed Gradient Descent in Distributed and Federated Optimizationcited by 90
  21. 2020Local SGD: Unified Theory and New Efficient Methodscited by 88
  22. 2019Communication-efficient distributed SGD with Sketchingcited by 81
  23. 2021MARINA: Faster Non-Convex Distributed Learning with Compressioncited by 77
  24. 2020Local Differential Privacy based Federated Learning for Internet of Thingscited by 74
  25. 2019Don't Jump Through Hoops and Remove Those Loops: SVRG and Katyusha are Better Without the Outer Loopcited by 68
  26. 2020Breaking the Communication-Privacy-Accuracy Trilemmacited by 68
  27. 2019Stochastic Distributed Learning with Gradient Quantization and Variance Reductioncited by 64
  28. 2017Efficient Private ERM for Smooth Objectivescited by 61
  29. 2018Distributed learning with compressed gradientscited by 53
  30. 2022BEER: Fast O(1/T) Rate for Decentralized Nonconvex Optimization with Communication Compressioncited by 41
  31. 2019Efficient Privacy-Preserving Stochastic Nonconvex Optimizationcited by 27
  32. 2020A Unified Analysis of Stochastic Gradient Methods for Nonconvex Federated Optimizationcited by 27
  33. 2021Lossless Compression of Efficient Private Local Randomizerscited by 27
  34. 20223PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregationcited by 27
  35. 2022Private Non-Convex Federated Learning Without a Trusted Servercited by 26
  36. 2021CANITA: Faster Rates for Distributed Convex Optimization with Communication Compressioncited by 23
  37. 2021FedPAGE: A Fast Local Stochastic Gradient Method for Communication-Efficient Federated Learningcited by 19
  38. 2020Private and Communication-Efficient Edge Learning: A Sparse Differential Gaussian-Masking Distributed SGD Approachcited by 18

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 10; 2024: 10; 2025: 7; 2026: 5.

34 papers.

Sort:

  1. 2024Advances and Open Challenges in Federated Foundation Modelscited by 33
  2. 2024A Survey on Efficient Federated Learning Methods for Foundation Model Trainingcited by 29
  3. 2023Privacy and Fairness in Federated Learning: on the Perspective of Trade-offcited by 26
  4. 2023Bayesian Federated Learning: A Surveycited by 22
  5. 2023A Survey for Federated Learning Evaluations: Goals and Measurescited by 16
  6. 2024OpenFGL: A Comprehensive Benchmark for Federated Graph Learningcited by 16
  7. 2022Coresets for Vertical Federated Learning: Regularized Linear Regression and K-Means Clusteringcited by 14
  8. 2023Differentially Private SGD Without Clipping Bias: An Error-Feedback Approachcited by 13
  9. 2024FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneitycited by 13
  10. 2023DIFF2: Differential Private Optimization via Gradient Differences for Nonconvex Distributed Learningcited by 10
  11. 2023Convergence and Privacy of Decentralized Nonconvex Optimization with Gradient Clipping and Communication Compressioncited by 9
  12. 2023A Communication-Efficient Adaptive Algorithm for Federated Learning under Cumulative Regretcited by 7
22 more
  1. 2022Simple and Optimal Stochastic Gradient Methods for Nonsmooth Nonconvex Optimizationcited by 5
  2. 2023Escaping Saddle Points in Heterogeneous Federated Learning via Distributed SGD with Communication Compressioncited by 4
  3. 2024PrivSGP-VR: Differentially Private Variance-Reduced Stochastic Gradient Push with Tight Utility Boundscited by 4
  4. 2024Communication-efficient Vertical Federated Learning via Compressed Error Feedbackcited by 3
  5. 2024Communication-Efficient Federated Group Distributionally Robust Optimizationcited by 3
  6. 2024Problem-dependent convergence bounds for randomized linear gradient compressioncited by 3
  7. 2025From Risk to Resilience: Towards Assessing and Mitigating the Risk of Data Reconstruction Attacks in Federated Learningcited by 3
  8. 2023Sparse Training for Federated Learning with Regularized Error Correctioncited by 2
  9. 2025FedSparQ: Adaptive Sparse Quantization with Error Feedback for Robust & Efficient Federated Learningcited by 2
  10. 2023Distributed Learning Meets 6G: A Communication and Computing Perspectivecited by 1
  11. 2024Federated Cubic Regularized Newton Learning with Sparsification-amplified Differential Privacycited by 1
  12. 2025Characterizing the Accuracy-Communication-Privacy Trade-off in Distributed Stochastic Convex Optimizationcited by 1
  13. 2025Smoothed Normalization for Efficient Distributed Private Optimizationcited by 1
  14. 2025DP-CSGP: Differentially Private Stochastic Gradient Push with Compressed Communicationcited by 1
  15. 2024FedCross: Intertemporal Federated Learning Under Evolutionary Gamescited by 0
  16. 2025Dyn-D2P: Dynamic Differentially Private Decentralized Learning with Provable Utility Guaranteecited by 0
  17. 2025First Provable Guarantees for Practical Private FL: Beyond Restrictive Assumptionscited by 0
  18. 2026ERIS: Enhancing Privacy and Scalability in Federated Learning via Federated Shard Aggregationcited by 0
  19. 2026On Gossip Algorithms for Machine Learning with Pairwise Objectivescited by 0
  20. 2026Scalable and Private Federated Learning Using Distributed Differential Privacy and Secure Aggregationcited by 0
  21. 2026SSTQ:Privacy-Preserving Vector Quantization via Subsampled Stochastic TurboQuantcited by 0
  22. 2026Communication-Efficient Agnostic Federated Learning via Faster Convergence and Compressioncited by 0