# Tailoring Gradient Methods for Differentially-Private Distributed Optimization

arXiv: https://arxiv.org/abs/2202.01113 · HTML: https://mlsanity.com/paper/2202.01113/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Feb 2022.

## Built on by (most cited first, 18 of 18)

- [Decentralized Stochastic Optimization with Inherent Privacy Protection](https://mlsanity.com/paper/2205.03884/) (2022)
- [Decentralized Nonconvex Optimization with Guaranteed Privacy and Accuracy](https://mlsanity.com/paper/2212.07534/) (2022)
- [On the Tradeoff between Privacy Preservation and Byzantine-Robustness in Decentralized Learning](https://mlsanity.com/paper/2308.14606/) (2023)
- [Privacy Attacks in Decentralized Learning](https://mlsanity.com/paper/2402.10001/) (2024)
- [PrivSGP-VR: Differentially Private Variance-Reduced Stochastic Gradient Push with Tight Utility Bounds](https://mlsanity.com/paper/2405.02638/) (2024)
- [A survey on secure decentralized optimization and learning](https://mlsanity.com/paper/2408.08628/) (2024)
- [Sparse Decentralized Federated Learning](https://mlsanity.com/paper/2308.16671/) (2023)
- [Locally Differentially Private Gradient Tracking for Distributed Online Learning over Directed Graphs](https://mlsanity.com/paper/2310.16105/) (2023)
- [Privacy-Preserving Distributed Optimization and Learning](https://mlsanity.com/paper/2403.00157/) (2024)
- [Generalizing Differentially Private Decentralized Deep Learning with Multi-Agent Consensus](https://mlsanity.com/paper/2306.13892/) (2023)
- [Locally Differentially Private Distributed Online Learning with Guaranteed Optimality](https://mlsanity.com/paper/2306.14094/) (2023)
- [Towards Trustworthy Federated Learning](https://mlsanity.com/paper/2503.03684/) (2025)
- [Distributed optimization: designed for federated learning](https://mlsanity.com/paper/2508.08606/) (2025)
- [DP-CSGP: Differentially Private Stochastic Gradient Push with Compressed Communication](https://mlsanity.com/paper/2512.13583/) (2025)
- [Dyn-D2P: Dynamic Differentially Private Decentralized Learning with Provable Utility Guarantee](https://mlsanity.com/paper/2505.06651/) (2025)
- [Distributed gradient methods under heavy-tailed communication noise](https://mlsanity.com/paper/2505.24464/) (2025)
- [ImprovDML: Improved Trade-off in Private Byzantine-Resilient Distributed Machine Learning](https://mlsanity.com/paper/2506.15181/) (2025)
- [A Weighted Gradient Tracking Privacy-Preserving Method for Distributed Optimization](https://mlsanity.com/paper/2509.18134/) (2025)

## Abstract (arXiv)

Decentralized optimization is gaining increased traction due to its widespread applications in large-scale machine learning and multi-agent systems. The same mechanism that enables its success, i.e., information sharing among participating agents, however, also leads to the disclosure of individual agents' private information, which is unacceptable when sensitive data are involved. As differential privacy is becoming a de facto standard for privacy preservation, recently results have emerged integrating differential privacy with distributed optimization. However, directly incorporating differential privacy design in existing distributed optimization approaches significantly compromises optimization accuracy. In this paper, we propose to redesign and tailor gradient methods for differentially-private distributed optimization, and propose two differential-privacy oriented gradient methods that can ensure both rigorous epsilon-differential privacy and optimality. The first algorithm is based on static-consensus based gradient methods, and the second algorithm is based on dynamic-consensus (gradient-tracking) based distributed optimization methods and, hence, is applicable to general directed interaction graph topologies. Both algorithms can simultaneously ensure almost sure convergence to an optimal solution and a finite privacy budget, even when the number of iterations goes to infinity. To our knowledge, this is the first time that both goals are achieved simultaneously. Numerical simulations using a distributed estimation problem and experimental results on a benchmark dataset confirm the effectiveness of the proposed approaches.