# Decentralized Stochastic Optimization with Inherent Privacy Protection arXiv: https://arxiv.org/abs/2205.03884 ยท HTML: https://mlsanity.com/paper/2205.03884/ ## Key facts - Cited by 11 papers in the map (data through 14 Oct 2026). - Builds on 28 of its 68 references that are in the map. - First posted on arXiv 8 May 2022. ## Builds on (most cited first, 28 of 28) - [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016) - [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016) - [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019) - [Exploiting Unintended Feature Leakage in Collaborative Learning](https://mlsanity.com/paper/1805.04049/) (2018) - [Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent](https://mlsanity.com/paper/1705.09056/) (2017) - [Differential Privacy Has Disparate Impact on Model Accuracy](https://mlsanity.com/paper/1905.12101/) (2019) - [Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis](https://mlsanity.com/paper/1702.03849/) (2017) - [Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication](https://mlsanity.com/paper/1902.00340/) (2019) - [EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization](https://mlsanity.com/paper/1404.6264/) (2014) - [Distributed Stochastic Gradient Tracking Methods](https://mlsanity.com/paper/1805.11454/) (2018) - [Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware](https://mlsanity.com/paper/1806.03287/) (2018) - [Differentially Private Model Publishing for Deep Learning](https://mlsanity.com/paper/1904.02200/) (2019) - [Stochastic Gradient-Push for Strongly Convex Functions on Time-Varying Directed Graphs](https://mlsanity.com/paper/1406.2075/) (2014) - [Communication-Efficient Distributed Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2003.06307/) (2020) - [Differentially Private Distributed Optimization](https://mlsanity.com/paper/1401.2596/) (2014) - [Tailoring Gradient Methods for Differentially-Private Distributed Optimization](https://mlsanity.com/paper/2202.01113/) (2022) - [Efficient Deep Learning on Multi-Source Private Data](https://mlsanity.com/paper/1807.06689/) (2018) - [Convergence rates for distributed stochastic optimization over random networks](https://mlsanity.com/paper/1803.07836/) (2018) - [Non-Convex Distributed Optimization](https://mlsanity.com/paper/1512.00895/) (2015) - [Secure and Privacy-Preserving Consensus](https://mlsanity.com/paper/1707.04491/) (2017) - [Privacy-Preserving Average Consensus via State Decomposition](https://mlsanity.com/paper/1902.09576/) (2019) - [Privacy Preservation in Distributed Subgradient Optimization Algorithms](https://mlsanity.com/paper/1512.08822/) (2015) - [Decentralized Consensus Algorithm with Delayed and Stochastic Gradients](https://mlsanity.com/paper/1604.05649/) (2016) - [Geometrically Convergent Distributed Optimization with Uncoordinated Step-Sizes](https://mlsanity.com/paper/1609.05877/) (2016) - [Hybrid Random/Deterministic Parallel Algorithms for Nonconvex Big Data Optimization](https://mlsanity.com/paper/1407.4504/) (2014) - [Stochastic Subgradient Algorithms for Strongly Convex Optimization over Distributed Networks](https://mlsanity.com/paper/1409.8277/) (2014) - [Distributed Stochastic Gradient Method for Non-Convex Problems with Applications in Supervised Learning](https://mlsanity.com/paper/1908.06693/) (2019) - [Algorithm-Level Confidentiality for Average Consensus on Time-Varying Directed Graphs](https://mlsanity.com/paper/2201.00293/) (2022) ## Built on by (most cited first, 11 of 11) - [Networked Signal and Information Processing](https://mlsanity.com/paper/2210.13767/) (2022) - [Decentralized Nonconvex Optimization with Guaranteed Privacy and Accuracy](https://mlsanity.com/paper/2212.07534/) (2022) - [On the Tradeoff between Privacy Preservation and Byzantine-Robustness in Decentralized Learning](https://mlsanity.com/paper/2308.14606/) (2023) - [Quantization enabled Privacy Protection in Decentralized Stochastic Optimization](https://mlsanity.com/paper/2208.04845/) (2022) - [Enforcing Privacy in Distributed Learning with Performance Guarantees](https://mlsanity.com/paper/2301.06412/) (2023) - [Quantization Avoids Saddle Points in Distributed Optimization](https://mlsanity.com/paper/2403.10423/) (2024) - [Communication Efficient Federated Learning with Linear Convergence on Heterogeneous Data](https://mlsanity.com/paper/2503.15804/) (2025) - [Immersion and Invariance-based Coding for Privacy-Preserving Federated Learning](https://mlsanity.com/paper/2409.17201/) (2024) - [Lossless Privacy-Preserving Aggregation for Decentralized Federated Learning](https://mlsanity.com/paper/2501.04409/) (2025) - [Gradient Purification: Defense Against Poisoning Attack in Decentralized Federated Learning](https://mlsanity.com/paper/2501.04453/) (2025) - [A Weighted Gradient Tracking Privacy-Preserving Method for Distributed Optimization](https://mlsanity.com/paper/2509.18134/) (2025) ## Abstract (arXiv) Decentralized stochastic optimization is the basic building block of modern collaborative machine learning, distributed estimation and control, and large-scale sensing. Since involved data usually contain sensitive information like user locations, healthcare records and financial transactions, privacy protection has become an increasingly pressing need in the implementation of decentralized stochastic optimization algorithms. In this paper, we propose a decentralized stochastic gradient descent algorithm which is embedded with inherent privacy protection for every participating agent against other participating agents and external eavesdroppers. This proposed algorithm builds in a dynamics based gradient-obfuscation mechanism to enable privacy protection without compromising optimization accuracy, which is in significant difference from differential-privacy based privacy solutions for decentralized optimization that have to trade optimization accuracy for privacy. The dynamics based privacy approach is encryption-free, and hence avoids incurring heavy communication or computation overhead, which is a common problem with encryption based privacy solutions for decentralized stochastic optimization. Besides rigorously characterizing the convergence performance of the proposed decentralized stochastic gradient descent algorithm under both convex objective functions and non-convex objective functions, we also provide rigorous information-theoretic analysis of its strength of privacy protection. Simulation results for a distributed estimation problem as well as numerical experiments for decentralized learning on a benchmark machine learning dataset confirm the effectiveness of the proposed approach.