# Quantization enabled Privacy Protection in Decentralized Stochastic Optimization arXiv: https://arxiv.org/abs/2208.04845 ยท HTML: https://mlsanity.com/paper/2208.04845/ ## Key facts - Cited by 7 papers in the map (data through 14 Oct 2026). - Builds on 34 of its 72 references that are in the map. - First posted on arXiv 7 Aug 2022. ## Builds on (most cited first, 30 of 34) - [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016) - [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019) - [Exploiting Unintended Feature Leakage in Collaborative Learning](https://mlsanity.com/paper/1805.04049/) (2018) - [Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent](https://mlsanity.com/paper/1705.09056/) (2017) - [The Composition Theorem for Differential Privacy](https://mlsanity.com/paper/1311.0776/) (2013) - [TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning](https://mlsanity.com/paper/1705.07878/) (2017) - [Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication](https://mlsanity.com/paper/1902.00340/) (2019) - [cpSGD: Communication-efficient and differentially-private distributed SGD](https://mlsanity.com/paper/1805.10559/) (2018) - [Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization](https://mlsanity.com/paper/1709.08765/) (2017) - [Distributed Stochastic Gradient Tracking Methods](https://mlsanity.com/paper/1805.11454/) (2018) - [Collaborative Deep Learning in Fixed Topology Networks](https://mlsanity.com/paper/1706.07880/) (2017) - [An Exact Quantized Decentralized Gradient Descent Algorithm](https://mlsanity.com/paper/1806.11536/) (2018) - [Stochastic Gradient-Push for Strongly Convex Functions on Time-Varying Directed Graphs](https://mlsanity.com/paper/1406.2075/) (2014) - [Differentially Private Distributed Optimization](https://mlsanity.com/paper/1401.2596/) (2014) - [Efficient Deep Learning on Multi-Source Private Data](https://mlsanity.com/paper/1807.06689/) (2018) - [Convergence rates for distributed stochastic optimization over random networks](https://mlsanity.com/paper/1803.07836/) (2018) - [Robust and Communication-Efficient Collaborative Learning](https://mlsanity.com/paper/1907.10595/) (2019) - [Decentralized Stochastic Optimization with Inherent Privacy Protection](https://mlsanity.com/paper/2205.03884/) (2022) - [Privacy-Preserving Distributed Optimization via Subspace Perturbation: A General Framework](https://mlsanity.com/paper/2004.13999/) (2020) - [Non-Convex Distributed Optimization](https://mlsanity.com/paper/1512.00895/) (2015) - [Secure and Privacy-Preserving Consensus](https://mlsanity.com/paper/1707.04491/) (2017) - [Privacy-Preserving Average Consensus via State Decomposition](https://mlsanity.com/paper/1902.09576/) (2019) - [Privacy Preservation in Distributed Subgradient Optimization Algorithms](https://mlsanity.com/paper/1512.08822/) (2015) - [Decentralized Consensus Algorithm with Delayed and Stochastic Gradients](https://mlsanity.com/paper/1604.05649/) (2016) - [Distributed Discrete-time Optimization in Multi-agent Networks Using only Sign of Relative State](https://mlsanity.com/paper/1709.08360/) (2017) - [Privacy preserving distributed optimization using homomorphic encryption](https://mlsanity.com/paper/1805.00572/) (2018) - [Decentralized Gradient Methods with Time-varying Uncoordinated Stepsizes: Convergence Analysis and Privacy Design](https://mlsanity.com/paper/2205.10934/) (2022) - [Quantization Design for Distributed Optimization](https://mlsanity.com/paper/1504.02317/) (2015) - [Cloud-based Quadratic Optimization with Partially Homomorphic Encryption](https://mlsanity.com/paper/1809.02267/) (2018) - [Recycled ADMM: Improving the Privacy and Accuracy of Distributed Algorithms](https://mlsanity.com/paper/1910.04581/) (2019) ## Built on by (most cited first, 7 of 7) - [Decentralized Nonconvex Optimization with Guaranteed Privacy and Accuracy](https://mlsanity.com/paper/2212.07534/) (2022) - [A survey on secure decentralized optimization and learning](https://mlsanity.com/paper/2408.08628/) (2024) - [Locally Differentially Private Gradient Tracking for Distributed Online Learning over Directed Graphs](https://mlsanity.com/paper/2310.16105/) (2023) - [Privacy-Preserving Distributed Optimization and Learning](https://mlsanity.com/paper/2403.00157/) (2024) - [Locally Differentially Private Distributed Online Learning with Guaranteed Optimality](https://mlsanity.com/paper/2306.14094/) (2023) - [QMGeo: Differentially Private Federated Learning via Stochastic Quantization with Mixed Truncated Geometric Distribution](https://mlsanity.com/paper/2312.05761/) (2023) - [DPQuant: Efficient and Differentially-Private Model Training via Dynamic Quantization Scheduling](https://mlsanity.com/paper/2509.03472/) (2025) ## Abstract (arXiv) By enabling multiple agents to cooperatively solve a global optimization problem in the absence of a central coordinator, decentralized stochastic optimization is gaining increasing attention in areas as diverse as machine learning, control, and sensor networks. Since the associated data usually contain sensitive information, such as user locations and personal identities, privacy protection has emerged as a crucial need in the implementation of decentralized stochastic optimization. In this paper, we propose a decentralized stochastic optimization algorithm that is able to guarantee provable convergence accuracy even in the presence of aggressive quantization errors that are proportional to the amplitude of quantization inputs. The result applies to both convex and non-convex objective functions, and enables us to exploit aggressive quantization schemes to obfuscate shared information, and hence enables privacy protection without losing provable optimization accuracy. In fact, by using a {stochastic} ternary quantization scheme, which quantizes any value to three numerical levels, we achieve quantization-based rigorous differential privacy in decentralized stochastic optimization, which has not been reported before. In combination with the presented quantization scheme, the proposed algorithm ensures, for the first time, rigorous differential privacy in decentralized stochastic optimization without losing provable convergence accuracy. Simulation results for a distributed estimation problem as well as numerical experiments for decentralized learning on a benchmark machine learning dataset confirm the effectiveness of the proposed approach.