# Differentially Private Distributed Estimation and Learning

arXiv: https://arxiv.org/abs/2306.15865 · HTML: https://mlsanity.com/paper/2306.15865/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 91 references that are in the map.
- First posted on arXiv 28 Jun 2023.

## Builds on (most cited first, 23 of 23)

- [A Hybrid Approach to Privacy-Preserving Federated Learning](https://mlsanity.com/paper/1812.03224/) (2018)
- [Federated Learning for Wireless Communications: Motivation, Opportunities and Challenges](https://mlsanity.com/paper/1908.06847/) (2019)
- [Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy](https://mlsanity.com/paper/2106.13673/) (2021)
- [Differentially Private SQL with Bounded User Contribution](https://mlsanity.com/paper/1909.01917/) (2019)
- [Fast Convergence Rates for Distributed Non-Bayesian Learning](https://mlsanity.com/paper/1508.05161/) (2015)
- [LinkedIn's Audience Engagements API: A Privacy Preserving Data Analytics System at Scale](https://mlsanity.com/paper/2002.05839/) (2020)
- [The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning](https://mlsanity.com/paper/2111.05326/) (2021)
- [Differential Privacy Techniques for Cyber Physical Systems: A Survey](https://mlsanity.com/paper/1812.02282/) (2018)
- [Social Learning and Distributed Hypothesis Testing](https://mlsanity.com/paper/1410.4307/) (2014)
- [Optimality of the Laplace Mechanism in Differential Privacy](https://mlsanity.com/paper/1504.00065/) (2015)
- [Differentially Private Histograms under Continual Observation: Streaming Selection into the Unknown](https://mlsanity.com/paper/2103.16787/) (2021)
- [Federated Data Analytics: A Study on Linear Models](https://mlsanity.com/paper/2206.07786/) (2022)
- [Distributed Detection : Finite-time Analysis and Impact of Network Topology](https://mlsanity.com/paper/1409.8606/) (2014)
- [Nonasymptotic Convergence Rates for Cooperative Learning Over Time-Varying Directed Graphs](https://mlsanity.com/paper/1410.1977/) (2014)
- [Private Weighted Sum Aggregation](https://mlsanity.com/paper/2010.10640/) (2020)
- [Joint Estimation and Localization in Sensor Networks](https://mlsanity.com/paper/1404.3580/) (2014)
- [Linear Time Average Consensus on Fixed Graphs and Implications for Decentralized Optimization and Multi-Agent Control](https://mlsanity.com/paper/1411.4186/) (2014)
- [Enforcing Privacy in Distributed Learning with Performance Guarantees](https://mlsanity.com/paper/2301.06412/) (2023)
- [Distributed Algorithms for Stochastic Source Seeking with Mobile Robot Networks: Technical Report](https://mlsanity.com/paper/1402.0051/) (2014)
- [Diffusing Private Data over Networks](https://mlsanity.com/paper/1511.06253/) (2015)
- [Bayesian Learning without Recall](https://mlsanity.com/paper/1601.06103/) (2016)
- [Distributed Estimation and Learning over Heterogeneous Networks](https://mlsanity.com/paper/1611.03328/) (2016)
- [Seeding with Differentially Private Network Information](https://mlsanity.com/paper/2305.16590/) (2023)

## Abstract (arXiv)

We study distributed estimation and learning problems in a networked environment where agents exchange information to estimate unknown statistical properties of random variables from their privately observed samples. The agents can collectively estimate the unknown quantities by exchanging information about their private observations, but they also face privacy risks. Our novel algorithms extend the existing distributed estimation literature and enable the participating agents to estimate a complete sufficient statistic from private signals acquired offline or online over time and to preserve the privacy of their signals and network neighborhoods. This is achieved through linear aggregation schemes with adjusted randomization schemes that add noise to the exchanged estimates subject to differential privacy (DP) constraints, both in an offline and online manner. We provide convergence rate analysis and tight finite-time convergence bounds. We show that the noise that minimizes the convergence time to the best estimates is the Laplace noise, with parameters corresponding to each agent's sensitivity to their signal and network characteristics. Our algorithms are amenable to dynamic topologies and balancing privacy and accuracy trade-offs. Finally, to supplement and validate our theoretical results, we run experiments on real-world data from the US Power Grid Network and electric consumption data from German Households to estimate the average power consumption of power stations and households under all privacy regimes and show that our method outperforms existing first-order, privacy-aware, distributed optimization methods.