# Differentially Private Simple Linear Regression

arXiv: https://arxiv.org/abs/2007.05157 · HTML: https://mlsanity.com/paper/2007.05157/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2020.

## Built on by (most cited first, 30 of 34)

- [Easy Differentially Private Linear Regression](https://mlsanity.com/paper/2208.07353/) (2022)
- [Private Gradient Descent for Linear Regression: Tighter Error Bounds and Instance-Specific Uncertainty Estimation](https://mlsanity.com/paper/2402.13531/) (2024)
- [Differentially Private Regression with Unbounded Covariates](https://mlsanity.com/paper/2202.11199/) (2022)
- [Near Optimal Private and Robust Linear Regression](https://mlsanity.com/paper/2301.13273/) (2023)
- [Mean Estimation with User-level Privacy under Data Heterogeneity](https://mlsanity.com/paper/2307.15835/) (2023)
- [Non-parametric Differentially Private Confidence Intervals for the Median](https://mlsanity.com/paper/2106.10333/) (2021)
- [Insufficient Statistics Perturbation: Stable Estimators for Private Least Squares](https://mlsanity.com/paper/2404.15409/) (2024)
- [Improved Differentially Private Regression via Gradient Boosting](https://mlsanity.com/paper/2303.03451/) (2023)
- [Optimal Unbiased Randomizers for Regression with Label Differential Privacy](https://mlsanity.com/paper/2312.05659/) (2023)
- [Differentially Private Post-Processing for Fair Regression](https://mlsanity.com/paper/2405.04034/) (2024)
- [Improved Analysis of Sparse Linear Regression in Local Differential Privacy Model](https://mlsanity.com/paper/2310.07367/) (2023)
- [Resampling methods for private statistical inference](https://mlsanity.com/paper/2402.07131/) (2024)
- [Differentially Private Nonparametric Regression Under a Growth Condition](https://mlsanity.com/paper/2111.12786/) (2021)
- [Differentially Private Estimation via Statistical Depth](https://mlsanity.com/paper/2207.12602/) (2022)
- [Personalized Differential Privacy for Ridge Regression](https://mlsanity.com/paper/2401.17127/) (2024)
- [Provable Privacy with Non-Private Pre-Processing](https://mlsanity.com/paper/2403.13041/) (2024)
- [Sample-Optimal Private Regression in Polynomial Time](https://mlsanity.com/paper/2503.24321/) (2025)
- [Differentially Private Distributed Bayesian Linear Regression with MCMC](https://mlsanity.com/paper/2301.13778/) (2023)
- [Label Differential Privacy via Aggregation](https://mlsanity.com/paper/2310.10092/) (2023)
- [Better Locally Private Sparse Estimation Given Multiple Samples Per User](https://mlsanity.com/paper/2408.04313/) (2024)
- [Differentially Private Linear Regression and Synthetic Data Generation with Statistical Guarantees](https://mlsanity.com/paper/2510.16974/) (2025)
- [Linear Model Against Malicious Adversaries with Local Differential Privacy](https://mlsanity.com/paper/2202.02448/) (2022)
- [Private Isotonic Regression](https://mlsanity.com/paper/2210.15175/) (2022)
- [On the Query Complexity of Training Data Reconstruction in Private Learning](https://mlsanity.com/paper/2303.16372/) (2023)
- [Pure Differential Privacy for Functional Summaries with a Laplace-like Process](https://mlsanity.com/paper/2309.00125/) (2023)
- [The Normal Distributions Indistinguishability Spectrum and its Application to Privacy-Preserving Machine Learning](https://mlsanity.com/paper/2309.01243/) (2023)
- [DP-KAN: Differentially Private Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2407.12569/) (2024)
- [Testing Credibility of Public and Private Surveys through the Lens of Regression](https://mlsanity.com/paper/2410.05458/) (2024)
- [Private Linear Regression with Differential Privacy and PAC Privacy](https://mlsanity.com/paper/2412.02578/) (2024)
- [Differentially Private Geodesic Regression](https://mlsanity.com/paper/2504.11304/) (2025)

## Abstract (arXiv)

Economics and social science research often require analyzing datasets of sensitive personal information at fine granularity, with models fit to small subsets of the data. Unfortunately, such fine-grained analysis can easily reveal sensitive individual information. We study algorithms for simple linear regression that satisfy differential privacy, a constraint which guarantees that an algorithm's output reveals little about any individual input data record, even to an attacker with arbitrary side information about the dataset. We consider the design of differentially private algorithms for simple linear regression for small datasets, with tens to hundreds of datapoints, which is a particularly challenging regime for differential privacy. Focusing on a particular application to small-area analysis in economics research, we study the performance of a spectrum of algorithms we adapt to the setting. We identify key factors that affect their performance, showing through a range of experiments that algorithms based on robust estimators (in particular, the Theil-Sen estimator) perform well on the smallest datasets, but that other more standard algorithms do better as the dataset size increases.