# Privacy-preserving Distributed Machine Learning via Local Randomization and ADMM Perturbation

arXiv: https://arxiv.org/abs/1908.01059 · HTML: https://mlsanity.com/paper/1908.01059/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jul 2019.

## Built on by (most cited first, 8 of 8)

- [SoK: Dataset Copyright Auditing in Machine Learning Systems](https://mlsanity.com/paper/2410.16618/) (2024)
- [Federated Stochastic Primal-dual Learning with Differential Privacy](https://mlsanity.com/paper/2204.12284/) (2022)
- [Linear Regression with Distributed Learning: A Generalization Error Perspective](https://mlsanity.com/paper/2101.09001/) (2021)
- [P4L: Privacy Preserving Peer-to-Peer Learning for Infrastructureless Setups](https://mlsanity.com/paper/2302.13438/) (2023)
- [Exact Penalty Method for Federated Learning](https://mlsanity.com/paper/2208.11231/) (2022)
- [FedSiam-DA: Dual-aggregated Federated Learning via Siamese Network under Non-IID Data](https://mlsanity.com/paper/2211.09421/) (2022)
- [Distributed Continual Learning with CoCoA in High-dimensional Linear Regression](https://mlsanity.com/paper/2312.01795/) (2023)
- [ADP-VRSGP: Decentralized Learning with Adaptive Differential Privacy via Variance-Reduced Stochastic Gradient Push](https://mlsanity.com/paper/2510.20157/) (2025)

## Abstract (arXiv)

With the proliferation of training data, distributed machine learning (DML) is becoming more competent for large-scale learning tasks. However, privacy concerns have to be given priority in DML, since training data may contain sensitive information of users. In this paper, we propose a privacy-preserving ADMM-based DML framework with two novel features: First, we remove the assumption commonly made in the literature that the users trust the server collecting their data. Second, the framework provides heterogeneous privacy for users depending on data's sensitive levels and servers' trust degrees. The challenging issue is to keep the accumulation of privacy losses over ADMM iterations minimal. In the proposed framework, a local randomization approach, which is differentially private, is adopted to provide users with self-controlled privacy guarantee for the most sensitive information. Further, the ADMM algorithm is perturbed through a combined noise-adding method, which simultaneously preserves privacy for users' less sensitive information and strengthens the privacy protection of the most sensitive information. We provide detailed analyses on the performance of the trained model according to its generalization error. Finally, we conduct extensive experiments using real-world datasets to validate the theoretical results and evaluate the classification performance of the proposed framework.