# The Value of Collaboration in Convex Machine Learning with Differential Privacy

arXiv: https://arxiv.org/abs/1906.09679 · HTML: https://mlsanity.com/paper/1906.09679/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2019.

## Built on by (most cited first, 24 of 24)

- [Improving LoRA in Privacy-preserving Federated Learning](https://mlsanity.com/paper/2403.12313/) (2024)
- [Private Federated Learning Without a Trusted Server: Optimal Algorithms for Convex Losses](https://mlsanity.com/paper/2106.09779/) (2021)
- [Private Non-Convex Federated Learning Without a Trusted Server](https://mlsanity.com/paper/2203.06735/) (2022)
- [Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2410.13097/) (2024)
- [FedSVD: Adaptive Orthogonalization for Private Federated Learning with LoRA](https://mlsanity.com/paper/2505.12805/) (2025)
- [Differentially Private Vertical Federated Clustering](https://mlsanity.com/paper/2208.01700/) (2022)
- [PPA: Preference Profiling Attack Against Federated Learning](https://mlsanity.com/paper/2202.04856/) (2022)
- [Continual Learning with Differential Privacy](https://mlsanity.com/paper/2110.05223/) (2021)
- [Differentially Private Federated Learning via Reconfigurable Intelligent Surface](https://mlsanity.com/paper/2203.17028/) (2022)
- [MUDGUARD: Taming Malicious Majorities in Federated Learning using Privacy-Preserving Byzantine-Robust Clustering](https://mlsanity.com/paper/2208.10161/) (2022)
- [PrivSGP-VR: Differentially Private Variance-Reduced Stochastic Gradient Push with Tight Utility Bounds](https://mlsanity.com/paper/2405.02638/) (2024)
- [Review of Mathematical Optimization in Federated Learning](https://mlsanity.com/paper/2412.01630/) (2024)
- [On the Practicality of Differential Privacy in Federated Learning by Tuning Iteration Times](https://mlsanity.com/paper/2101.04163/) (2021)
- [Utility Fairness for the Differentially Private Federated Learning](https://mlsanity.com/paper/2109.05267/) (2021)
- [Theoretical Analysis of Privacy Leakage in Trustworthy Federated Learning: A Perspective from Linear Algebra and Optimization Theory](https://mlsanity.com/paper/2407.16735/) (2024)
- [The Power of Bias: Optimizing Client Selection in Federated Learning with Heterogeneous Differential Privacy](https://mlsanity.com/paper/2408.08642/) (2024)
- [Optimizing the Numbers of Queries and Replies in Federated Learning with Differential Privacy](https://mlsanity.com/paper/2107.01895/) (2021)
- [User-Entity Differential Privacy in Learning Natural Language Models](https://mlsanity.com/paper/2211.01141/) (2022)
- [Locally Differentially Private Distributed Online Learning with Guaranteed Optimality](https://mlsanity.com/paper/2306.14094/) (2023)
- [DP-CSGP: Differentially Private Stochastic Gradient Push with Compressed Communication](https://mlsanity.com/paper/2512.13583/) (2025)
- [Two-Bit Aggregation for Communication Efficient and Differentially Private Federated Learning](https://mlsanity.com/paper/2110.03017/) (2021)
- [Dyn-D2P: Dynamic Differentially Private Decentralized Learning with Provable Utility Guarantee](https://mlsanity.com/paper/2505.06651/) (2025)
- [Privacy-Preserving Federated Learning from Partial Decryption Verifiable Threshold Multi-Client Functional Encryption](https://mlsanity.com/paper/2511.12936/) (2025)
- [Decoupling Identity from Utility: Privacy-by-Design Frameworks for Financial Ecosystems](https://mlsanity.com/paper/2604.14495/) (2026)

## Abstract (arXiv)

In this paper, we apply machine learning to distributed private data owned by multiple data owners, entities with access to non-overlapping training datasets. We use noisy, differentially-private gradients to minimize the fitness cost of the machine learning model using stochastic gradient descent. We quantify the quality of the trained model, using the fitness cost, as a function of privacy budget and size of the distributed datasets to capture the trade-off between privacy and utility in machine learning. This way, we can predict the outcome of collaboration among privacy-aware data owners prior to executing potentially computationally-expensive machine learning algorithms. Particularly, we show that the difference between the fitness of the trained machine learning model using differentially-private gradient queries and the fitness of the trained machine model in the absence of any privacy concerns is inversely proportional to the size of the training datasets squared and the privacy budget squared. We successfully validate the performance prediction with the actual performance of the proposed privacy-aware learning algorithms, applied to: financial datasets for determining interest rates of loans using regression; and detecting credit card frauds using support vector machines.