# Privacy Amplification by Iteration for ADMM with (Strongly) Convex Objective Functions

arXiv: https://arxiv.org/abs/2312.08685 · HTML: https://mlsanity.com/paper/2312.08685/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 53 references that are in the map.
- First posted on arXiv 14 Dec 2023.

## Builds on (most cited first, 27 of 27)

- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds](https://mlsanity.com/paper/1605.02065/) (2016)
- [Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences](https://mlsanity.com/paper/1807.01647/) (2018)
- [Subsampled R'enyi Differential Privacy and Analytical Moments Accountant](https://mlsanity.com/paper/1808.00087/) (2018)
- [Amplification by Shuffling: From Local to Central Differential Privacy via Anonymity](https://mlsanity.com/paper/1811.12469/) (2018)
- [Distributed Differential Privacy via Shuffling](https://mlsanity.com/paper/1808.01394/) (2018)
- [Privacy Amplification by Iteration](https://mlsanity.com/paper/1808.06651/) (2018)
- [The Privacy Blanket of the Shuffle Model](https://mlsanity.com/paper/1903.02837/) (2019)
- [Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descent](https://mlsanity.com/paper/2102.05855/) (2021)
- [Differentially Private Release and Learning of Threshold Functions](https://mlsanity.com/paper/1504.07553/) (2015)
- [Differentially Private Learning Needs Hidden State (Or Much Faster Convergence)](https://mlsanity.com/paper/2203.05363/) (2022)
- [DP-ADMM: ADMM-based Distributed Learning with Differential Privacy](https://mlsanity.com/paper/1808.10101/) (2018)
- [Lottery Ticket Preserves Weight Correlation: Is It Desirable or Not?](https://mlsanity.com/paper/2102.11068/) (2021)
- [Improving the Privacy and Accuracy of ADMM-Based Distributed Algorithms](https://mlsanity.com/paper/1806.02246/) (2018)
- [Privacy Amplification by Mixing and Diffusion Mechanisms](https://mlsanity.com/paper/1905.12264/) (2019)
- [Characterizing the Sample Complexity of Private Learners](https://mlsanity.com/paper/1402.2224/) (2014)
- [NxMTransformer: Semi-Structured Sparsification for Natural Language Understanding via ADMM](https://mlsanity.com/paper/2110.15766/) (2021)
- [An Accelerated Linearized Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1401.6607/) (2014)
- [Neural Architecture Dilation for Adversarial Robustness](https://mlsanity.com/paper/2108.06885/) (2021)
- [Scalable Intervention Target Estimation in Linear Models](https://mlsanity.com/paper/2111.07512/) (2021)
- [Improving Privacy-Preserving Vertical Federated Learning by Efficient Communication with ADMM](https://mlsanity.com/paper/2207.10226/) (2022)
- [Stochastic Variance-Reduced ADMM](https://mlsanity.com/paper/1604.07070/) (2016)
- [Differentially Private ADMM Algorithms for Machine Learning](https://mlsanity.com/paper/2011.00164/) (2020)
- [Faster Algorithms for Learning Convex Functions](https://mlsanity.com/paper/2111.01348/) (2021)
- [From Noisy Fixed-Point Iterations to Private ADMM for Centralized and Federated Learning](https://mlsanity.com/paper/2302.12559/) (2023)
- [Fast Stochastic Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1308.3558/) (2013)
- [Renyi Differentially Private ADMM for Non-Smooth Regularized Optimization](https://mlsanity.com/paper/1909.08180/) (2019)

## Built on by (most cited first, 1 of 1)

- [Fast John Ellipsoid Computation with Differential Privacy Optimization](https://mlsanity.com/paper/2408.06395/) (2024)

## Abstract (arXiv)

We examine a private ADMM variant for (strongly) convex objectives which is a primal-dual iterative method. Each iteration has a user with a private function used to update the primal variable, masked by Gaussian noise for local privacy, without directly adding noise to the dual variable. Privacy amplification by iteration explores if noises from later iterations can enhance the privacy guarantee when releasing final variables after the last iteration. Cyffers et al. [ICML 2023] explored privacy amplification by iteration for the proximal ADMM variant, where a user's entire private function is accessed and noise is added to the primal variable. In contrast, we examine a private ADMM variant requiring just one gradient access to a user's function, but both primal and dual variables must be passed between successive iterations. To apply Balle et al.'s [NeurIPS 2019] coupling framework to the gradient ADMM variant, we tackle technical challenges with novel ideas. First, we address the non-expansive mapping issue in ADMM iterations by using a customized norm. Second, because the dual variables are not masked with any noise directly, their privacy guarantees are achieved by treating two consecutive noisy ADMM iterations as a Markov operator. Our main result is that the privacy guarantee for the gradient ADMM variant can be amplified proportionally to the number of iterations. For strongly convex objective functions, this amplification exponentially increases with the number of iterations. These amplification results align with the previously studied special case of stochastic gradient descent.