mlsanityData to 7 Oct 2026

From Noisy Fixed-Point Iterations to Private ADMM for Centralized and Federated Learning

arXiv 2302.12559 ↗First posted 24 Feb 2023cs.LG cs.CR cs.DCcited by 5

Not on the map. Why?

Abstract (arXiv)

We study differentially private (DP) machine learning algorithms as instances of noisy fixed-point iterations, in order to derive privacy and utility results from this well-studied framework. We show that this new perspective recovers popular private gradient-based methods like DP-SGD and provides a principled way to design and analyze new private optimization algorithms in a flexible manner. Focusing on the widely-used Alternating Directions Method of Multipliers (ADMM) method, we use our general framework to derive novel private ADMM algorithms for centralized, federated and fully decentralized learning. For these three algorithms, we establish strong privacy guarantees leveraging privacy amplification by iteration and by subsampling. Finally, we provide utility guarantees using a unified analysis that exploits a recent linear convergence result for noisy fixed-point iterations.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 36 of its 61 references that are in the map. First posted on arXiv 24 Feb 2023.

Builds on

36 of 61 references.

References · 36

  1. 2019Advances and Open Problems in Federated Learningcited by 3,021
  2. 2016Deep Learning with Differential Privacycited by 2,682
  3. 2017Renyi Differential Privacycited by 643
  4. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
  5. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  6. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
  7. 2018Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergencescited by 237
  8. 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 199
  9. 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 169
  10. 2018Differentially Private Empirical Risk Minimization Revisited: Faster and More Generalcited by 127
  11. 2014Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Boundscited by 104
  12. 2018Privacy Amplification by Iterationcited by 97
24 more
  1. 2020Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identificationcited by 88
  2. 2021Differentially Private Federated Learning on Heterogeneous Datacited by 74
  3. 2016Decentralized Collaborative Learning of Personalized Models over Networkscited by 73
  4. 2022Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Losscited by 60
  5. 2013On the Linear Convergence of the ADMM in Decentralized Consensus Optimizationcited by 57
  6. 2022Federated Learning via Inexact ADMMcited by 44
  7. 2018DP-ADMM: ADMM-based Distributed Learning with Differential Privacycited by 41
  8. 2018Walkman: A Communication-Efficient Random-Walk Algorithm for Decentralized Optimizationcited by 30
  9. 2020Privacy Amplification by Decentralizationcited by 28
  10. 2022Muffliato: Peer-to-Peer Privacy Amplification for Decentralized Optimization and Averagingcited by 27
  11. 2018Improving the Privacy and Accuracy of ADMM-Based Distributed Algorithmscited by 20
  12. 2020Fixed Point Strategies in Data Sciencecited by 16
  13. 2019Learning Privately over Distributed Features: An ADMM Sharing Approachcited by 14
  14. 2018Operator Splitting Performance Estimation: Tight contraction factors and optimal parameter selectioncited by 13
  15. 2013On the O(1/k) Convergence of Asynchronous Distributed Alternating Direction Method of Multiplierscited by 11
  16. 2021Differentially Private Federated Learning via Inexact ADMMcited by 11
  17. 2021Differentially Private Coordinate Descent for Composite Empirical Risk Minimizationcited by 10
  18. 2013Asynchronous Distributed Optimization using a Randomized Alternating Direction Method of Multiplierscited by 9
  19. 2014Convergence Rates with Inexact Non-expansive Operatorscited by 8
  20. 2020Differentially Private ADMM Algorithms for Machine Learningcited by 5
  21. 2014A Coordinate Descent Primal-Dual Algorithm and Application to Distributed Asynchronous Optimizationcited by 3
  22. 2016Line Search for Averaged Operator Iterationcited by 3
  23. 2017Stochastic Quasi-Fej'er Block-Coordinate Fixed Point Iterations With Random Sweeping II: Mean-Square and Linear Convergencecited by 3
  24. 2020Towards Plausible Differentially Private ADMM Based Distributed Machine Learningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 0; 2026: 1.

5 papers.

  1. 2024Differentially Private Decentralized Learning with Random Walkscited by 8
  2. 2024Privacy Attacks in Decentralized Learningcited by 8
  3. 2023Privacy Amplification by Iteration for ADMM with (Strongly) Convex Objective Functionscited by 1
  4. 2024History-Aware and Dynamic Client Contribution in Federated Learningcited by 0
  5. 2026SPADE-DFL: Communication-Efficient Decentralized Federated Learning via Derivative-Free Linearized ADMMcited by 0