# On the Importance of Architecture and Feature Selection in Differentially Private Machine Learning

arXiv: https://arxiv.org/abs/2205.06720 · HTML: https://mlsanity.com/paper/2205.06720/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 63 references that are in the map.
- First posted on arXiv 13 May 2022.

## Builds on (most cited first, 18 of 18)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Differential Privacy Has Disparate Impact on Model Accuracy](https://mlsanity.com/paper/1905.12101/) (2019)
- [Differentially Private Learning Needs Better Features (or Much More Data)](https://mlsanity.com/paper/2011.11660/) (2020)
- [SGD: General Analysis and Improved Rates](https://mlsanity.com/paper/1901.09401/) (2019)
- [Understanding Gradient Clipping in Private SGD: A Geometric Perspective](https://mlsanity.com/paper/2006.15429/) (2020)
- [Genetic CNN](https://mlsanity.com/paper/1703.01513/) (2017)
- [Tempered Sigmoid Activations for Deep Learning with Differential Privacy](https://mlsanity.com/paper/2007.14191/) (2020)
- [Hyperparameter Tuning with Renyi Differential Privacy](https://mlsanity.com/paper/2110.03620/) (2021)
- [Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identification](https://mlsanity.com/paper/2007.03813/) (2020)
- [Segmentations-Leak: Membership Inference Attacks and Defenses in Semantic Image Segmentation](https://mlsanity.com/paper/1912.09685/) (2019)
- [Improving Deep Learning with Differential Privacy using Gradient Encoding and Denoising](https://mlsanity.com/paper/2007.11524/) (2020)
- [Three Tools for Practical Differential Privacy](https://mlsanity.com/paper/1812.02890/) (2018)
- [DPNAS: Neural Architecture Search for Deep Learning with Differential Privacy](https://mlsanity.com/paper/2110.08557/) (2021)
- [Automatic Discovery of Privacy-Utility Pareto Fronts](https://mlsanity.com/paper/1905.10862/) (2019)
- [Removing Disparate Impact of Differentially Private Stochastic Gradient Descent on Model Accuracy](https://mlsanity.com/paper/2003.03699/) (2020)
- [Architecture Matters: Investigating the Influence of Differential Privacy on Neural Network Design](https://mlsanity.com/paper/2111.14924/) (2021)

## Built on by (most cited first, 3 of 3)

- [Closed-Form Bounds for DP-SGD against Record-level Inference](https://mlsanity.com/paper/2402.14397/) (2024)
- [FIBER: A Differentially Private Optimizer with Filter-Aware Innovation Bias Correction](https://mlsanity.com/paper/2605.03425/) (2026)
- [Towards Reliable and Generalizable Differentially Private Machine Learning (Extended Version)](https://mlsanity.com/paper/2508.15141/) (2025)

## Abstract (arXiv)

We study a pitfall in the typical workflow for differentially private machine learning. The use of differentially private learning algorithms in a "drop-in" fashion -- without accounting for the impact of differential privacy (DP) noise when choosing what feature engineering operations to use, what features to select, or what neural network architecture to use -- yields overly complex and poorly performing models. In other words, by anticipating the impact of DP noise, a simpler and more accurate alternative model could have been trained for the same privacy guarantee. We systematically study this phenomenon through theory and experiments. On the theory front, we provide an explanatory framework and prove that the phenomenon arises naturally from the addition of noise to satisfy differential privacy. On the experimental front, we demonstrate how the phenomenon manifests in practice using various datasets, types of models, tasks, and neural network architectures. We also analyze the factors that contribute to the problem and distill our experimental insights into concrete takeaways that practitioners can follow when training models with differential privacy. Finally, we propose privacy-aware algorithms for feature selection and neural network architecture search. We analyze their differential privacy properties and evaluate them empirically.