# Better Private Linear Regression Through Better Private Feature Selection

arXiv: https://arxiv.org/abs/2306.00920 · HTML: https://mlsanity.com/paper/2306.00920/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 13 of 13)

- [Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identification](https://mlsanity.com/paper/2007.03813/) (2020)
- [Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domain](https://mlsanity.com/paper/1803.02596/) (2018)
- [When Does Differentially Private Learning Not Suffer in High Dimensions?](https://mlsanity.com/paper/2207.00160/) (2022)
- [Covariance-Aware Private Mean Estimation Without Private Covariance Estimation](https://mlsanity.com/paper/2106.13329/) (2021)
- [Practical Differentially Private Top-k Selection with Pay-what-you-get Composition](https://mlsanity.com/paper/1905.04273/) (2019)
- [Easy Differentially Private Linear Regression](https://mlsanity.com/paper/2208.07353/) (2022)
- [Private Multiplicative Weights Beyond Linear Queries](https://mlsanity.com/paper/1407.1571/) (2014)
- [Private Causal Inference](https://mlsanity.com/paper/1512.05469/) (2015)
- [A Joint Exponential Mechanism For Differentially Private Top-k](https://mlsanity.com/paper/2201.12333/) (2022)
- [Towards Instance-Optimal Private Query Release](https://mlsanity.com/paper/1811.03763/) (2018)
- [Differentially Private Algorithms for Empirical Machine Learning](https://mlsanity.com/paper/1411.5428/) (2014)
- [Improved Differentially Private Regression via Gradient Boosting](https://mlsanity.com/paper/2303.03451/) (2023)
- [The Kendall Interaction Filter for Variable Interaction Screening in Ultra High Dimensional Classification Problems](https://mlsanity.com/paper/2010.06688/) (2020)

## Built on by (most cited first, 5 of 5)

- [Tukey Depth Mechanisms for Practical Private Mean Estimation](https://mlsanity.com/paper/2502.18698/) (2025)
- [Differentially Private Linear Regression and Synthetic Data Generation with Statistical Guarantees](https://mlsanity.com/paper/2510.16974/) (2025)
- [Adaptive Power Iteration Method for Differentially Private PCA](https://mlsanity.com/paper/2602.11454/) (2026)
- [Private Linear Regression with Differential Privacy and PAC Privacy](https://mlsanity.com/paper/2412.02578/) (2024)
- [Outsourced Privacy-Preserving Feature Selection Based on Fully Homomorphic Encryption](https://mlsanity.com/paper/2505.12869/) (2025)

## Abstract (arXiv)

Existing work on differentially private linear regression typically assumes that end users can precisely set data bounds or algorithmic hyperparameters. End users often struggle to meet these requirements without directly examining the data (and violating privacy). Recent work has attempted to develop solutions that shift these burdens from users to algorithms, but they struggle to provide utility as the feature dimension grows. This work extends these algorithms to higher-dimensional problems by introducing a differentially private feature selection method based on Kendall rank correlation. We prove a utility guarantee for the setting where features are normally distributed and conduct experiments across 25 datasets. We find that adding this private feature selection step before regression significantly broadens the applicability of ``plug-and-play'' private linear regression algorithms at little additional cost to privacy, computation, or decision-making by the end user.