# SoK: A Review of Differentially Private Linear Models For High-Dimensional Data

arXiv: https://arxiv.org/abs/2404.01141 · HTML: https://mlsanity.com/paper/2404.01141/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 66 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 26 of 26)

- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds](https://mlsanity.com/paper/1605.02065/) (2016)
- [The Composition Theorem for Differential Privacy](https://mlsanity.com/paper/1311.0776/) (2013)
- [Coordinate Descent Algorithms](https://mlsanity.com/paper/1502.04759/) (2015)
- [A Rewriting System for Convex Optimization Problems](https://mlsanity.com/paper/1709.04494/) (2017)
- [Best Subset Selection via a Modern Optimization Lens](https://mlsanity.com/paper/1507.03133/) (2015)
- [Private Stochastic Convex Optimization: Optimal Rates in ℓ1 Geometry](https://mlsanity.com/paper/2103.01516/) (2021)
- [Private Learning and Sanitization: Pure vs. Approximate Differential Privacy](https://mlsanity.com/paper/1407.2674/) (2014)
- [Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization](https://mlsanity.com/paper/1710.05179/) (2017)
- [On Iterative Hard Thresholding Methods for High-dimensional M-Estimation](https://mlsanity.com/paper/1410.5137/) (2014)
- [Non-Euclidean Differentially Private Stochastic Convex Optimization: Optimal Rates in Linear Time](https://mlsanity.com/paper/2103.01278/) (2021)
- [High Dimensional Differentially Private Stochastic Optimization with Heavy-tailed Data](https://mlsanity.com/paper/2107.11136/) (2021)
- [Differential Privacy in the Shuffle Model: A Survey of Separations](https://mlsanity.com/paper/2107.11839/) (2021)
- [Collect at Once, Use Effectively: Making Non-interactive Locally Private Learning Possible](https://mlsanity.com/paper/1706.03316/) (2017)
- [On the Efficacy of Differentially Private Few-shot Image Classification](https://mlsanity.com/paper/2302.01190/) (2023)
- [Scaling Up Differentially Private LASSO Regularized Logistic Regression via Faster Frank-Wolfe Iterations](https://mlsanity.com/paper/2310.19978/) (2023)
- [The Challenge of Differentially Private Screening Rules](https://mlsanity.com/paper/2303.10303/) (2023)
- [Differential Privacy for Sparse Classification Learning](https://mlsanity.com/paper/1908.00780/) (2019)
- [A Knowledge Transfer Framework for Differentially Private Sparse Learning](https://mlsanity.com/paper/1909.06322/) (2019)
- [Private Federated Statistics in an Interactive Setting](https://mlsanity.com/paper/2211.10082/) (2022)
- [High-Dimensional Private Empirical Risk Minimization by Greedy Coordinate Descent](https://mlsanity.com/paper/2207.01560/) (2022)
- [Probing the Transition to Dataset-Level Privacy in ML Models Using an Output-Specific and Data-Resolved Privacy Profile](https://mlsanity.com/paper/2306.15790/) (2023)
- [A novel analysis of utility in privacy pipelines, using Kronecker products and quantitative information flow](https://mlsanity.com/paper/2308.11110/) (2023)

## Built on by (most cited first, 5 of 5)

- [High-Dimensional Distributed Sparse Classification with Scalable Communication-Efficient Global Updates](https://mlsanity.com/paper/2407.06346/) (2024)
- [More Options for Prelabor Rupture of Membranes, A Bayesian Analysis](https://mlsanity.com/paper/2408.10876/) (2024)
- [Adversarial Machine Learning Attacks on Financial Reporting via Maximum Violated Multi-Objective Attack](https://mlsanity.com/paper/2507.05441/) (2025)
- [Position: Challenges and Opportunities for Differential Privacy in the U.S. Federal Government](https://mlsanity.com/paper/2410.16423/) (2024)
- [AI Security Research Should Better Incentivize Defense Research](https://mlsanity.com/paper/2605.23448/) (2026)

## Abstract (arXiv)

Linear models are ubiquitous in data science, but are particularly prone to overfitting and data memorization in high dimensions. To guarantee the privacy of training data, differential privacy can be used. Many papers have proposed optimization techniques for high-dimensional differentially private linear models, but a systematic comparison between these methods does not exist. We close this gap by providing a comprehensive review of optimization methods for private high-dimensional linear models. Empirical tests on all methods demonstrate robust and coordinate-optimized algorithms perform best, which can inform future research. Code for implementing all methods is released online.