# Privacy-Utility Trade-off of Linear Regression under Random Projections and Additive Noise

arXiv: https://arxiv.org/abs/1902.04688 · HTML: https://mlsanity.com/paper/1902.04688/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Feb 2019.

## Built on by (most cited first, 13 of 13)

- [An Introduction to Johnson-Lindenstrauss Transforms](https://mlsanity.com/paper/2103.00564/) (2021)
- [Stochastic Coded Federated Learning with Convergence and Privacy Guarantees](https://mlsanity.com/paper/2201.10092/) (2022)
- [Distributed Sketching for Randomized Optimization: Exact Characterization, Concentration and Lower Bounds](https://mlsanity.com/paper/2203.09755/) (2022)
- [Stochastic Coded Federated Learning: Theoretical Analysis and Incentive Mechanism Design](https://mlsanity.com/paper/2211.04132/) (2022)
- [LoRA Provides Differential Privacy by Design via Random Sketching](https://mlsanity.com/paper/2409.17538/) (2024)
- [Video-DPRP: A Differentially Private Approach for Visual Privacy-Preserving Video Human Activity Recognition](https://mlsanity.com/paper/2503.02132/) (2025)
- [Dynamic Network-Assisted D2D-Aided Coded Distributed Learning](https://mlsanity.com/paper/2111.14789/) (2021)
- [The Gaussian Mixing Mechanism: Renyi Differential Privacy via Gaussian Sketches](https://mlsanity.com/paper/2505.24603/) (2025)
- [Adaptive Coded Federated Learning: Privacy Preservation and Straggler Mitigation](https://mlsanity.com/paper/2403.14905/) (2024)
- [Iterative Sketching for Secure Coded Regression](https://mlsanity.com/paper/2308.04185/) (2023)
- [The Normal Distributions Indistinguishability Spectrum and its Application to Privacy-Preserving Machine Learning](https://mlsanity.com/paper/2309.01243/) (2023)
- [Deep Learning with Data Privacy via Residual Perturbation](https://mlsanity.com/paper/2408.05723/) (2024)
- [Privacy-Utility Tradeoffs in Quantum Information Processing](https://mlsanity.com/paper/2602.10510/) (2026)

## Abstract (arXiv)

Data privacy is an important concern in machine learning, and is fundamentally at odds with the task of training useful learning models, which typically require the acquisition of large amounts of private user data. One possible way of fulfilling the machine learning task while preserving user privacy is to train the model on a transformed, noisy version of the data, which does not reveal the data itself directly to the training procedure. In this work, we analyze the privacy-utility trade-off of two such schemes for the problem of linear regression: additive noise, and random projections. In contrast to previous work, we consider a recently proposed notion of differential privacy that is based on conditional mutual information (MI-DP), which is stronger than the conventional (ε, δ)-differential privacy, and use relative objective error as the utility metric. We find that projecting the data to a lower-dimensional subspace before adding noise attains a better trade-off in general. We also make a connection between privacy problem and (non-coherent) SIMO, which has been extensively studied in wireless communication, and use tools from there for the analysis. We present numerical results demonstrating the performance of the schemes.