# Feature Selection from Differentially Private Correlations

arXiv: https://arxiv.org/abs/2408.10862 · HTML: https://mlsanity.com/paper/2408.10862/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 62 references that are in the map.
- First posted on arXiv 20 Aug 2024.

## Builds on (most cited first, 12 of 12)

- [Gap Safe screening rules for sparsity enforcing penalties](https://mlsanity.com/paper/1611.05780/) (2016)
- [Practical Differentially Private Top-k Selection with Pay-what-you-get Composition](https://mlsanity.com/paper/1905.04273/) (2019)
- [Forward-Backward Selection with Early Dropping](https://mlsanity.com/paper/1705.10770/) (2017)
- [Oneshot Differentially Private Top-k Selection](https://mlsanity.com/paper/2105.08233/) (2021)
- [Screening Tests for Lasso Problems](https://mlsanity.com/paper/1405.4897/) (2014)
- [Screening Rules for Lasso with Non-Convex Sparse Regularizers](https://mlsanity.com/paper/1902.06125/) (2019)
- [Scaling Up Differentially Private LASSO Regularized Logistic Regression via Faster Frank-Wolfe Iterations](https://mlsanity.com/paper/2310.19978/) (2023)
- [The Challenge of Differentially Private Screening Rules](https://mlsanity.com/paper/2303.10303/) (2023)
- [Fast and Scalable Lasso via Stochastic Frank-Wolfe Methods with a Convergence Guarantee](https://mlsanity.com/paper/1510.07169/) (2015)
- [Look-Ahead Screening Rules for the Lasso](https://mlsanity.com/paper/2105.05648/) (2021)
- [Differentially Private Top-k Selection via Canonical Lipschitz Mechanism](https://mlsanity.com/paper/2201.13376/) (2022)
- [Provable Privacy with Non-Private Pre-Processing](https://mlsanity.com/paper/2403.13041/) (2024)

## Built on by (most cited first, 2 of 2)

- [Differentially Private Iterative Screening Rules for Linear Regression](https://mlsanity.com/paper/2502.18578/) (2025)
- [Position: Challenges and Opportunities for Differential Privacy in the U.S. Federal Government](https://mlsanity.com/paper/2410.16423/) (2024)

## Abstract (arXiv)

Data scientists often seek to identify the most important features in high-dimensional datasets. This can be done through L1-regularized regression, but this can become inefficient for very high-dimensional datasets. Additionally, high-dimensional regression can leak information about individual datapoints in a dataset. In this paper, we empirically evaluate the established baseline method for feature selection with differential privacy, the two-stage selection technique, and show that it is not stable under sparsity. This makes it perform poorly on real-world datasets, so we consider a different approach to private feature selection. We employ a correlations-based order statistic to choose important features from a dataset and privatize them to ensure that the results do not leak information about individual datapoints. We find that our method significantly outperforms the established baseline for private feature selection on many datasets.