# DP-HyPO: An Adaptive Private Hyperparameter Optimization Framework

arXiv: https://arxiv.org/abs/2306.05734 · HTML: https://mlsanity.com/paper/2306.05734/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 43 references that are in the map.
- First posted on arXiv 9 Jun 2023.

## Builds on (most cited first, 23 of 23)

- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization](https://mlsanity.com/paper/1603.06560/) (2016)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [AutoML: A Survey of the State-of-the-Art](https://mlsanity.com/paper/1908.00709/) (2019)
- [The Composition Theorem for Differential Privacy](https://mlsanity.com/paper/1311.0776/) (2013)
- [Random Search and Reproducibility for Neural Architecture Search](https://mlsanity.com/paper/1902.07638/) (2019)
- [Unlocking High-Accuracy Differentially Private Image Classification through Scale](https://mlsanity.com/paper/2204.13650/) (2022)
- [Neural Architecture Search with Bayesian Optimisation and Optimal Transport](https://mlsanity.com/paper/1802.07191/) (2018)
- [Hyper-Parameter Optimization: A Review of Algorithms and Applications](https://mlsanity.com/paper/2003.05689/) (2020)
- [Hyperparameter Tuning with Renyi Differential Privacy](https://mlsanity.com/paper/2110.03620/) (2021)
- [Large Scale Private Learning via Low-rank Reparametrization](https://mlsanity.com/paper/2106.09352/) (2021)
- [Private Selection from Private Candidates](https://mlsanity.com/paper/1811.07971/) (2018)
- [Private Hypothesis Selection](https://mlsanity.com/paper/1905.13229/) (2019)
- [Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search](https://mlsanity.com/paper/1807.06906/) (2018)
- [The Role of Adaptive Optimizers for Honest Private Hyperparameter Selection](https://mlsanity.com/paper/2111.04906/) (2021)
- [sharpDARTS: Faster and More Accurate Differentiable Architecture Search](https://mlsanity.com/paper/1903.09900/) (2019)
- [Deep Active Learning with a Neural Architecture Search](https://mlsanity.com/paper/1811.07579/) (2018)
- [Locally Private Hypothesis Selection](https://mlsanity.com/paper/2002.09465/) (2020)
- [Wide Network Learning with Differential Privacy](https://mlsanity.com/paper/2103.01294/) (2021)
- [Towards modular and programmable architecture search](https://mlsanity.com/paper/1909.13404/) (2019)
- [Generalized Private Selection and Testing with High Confidence](https://mlsanity.com/paper/2211.12063/) (2022)
- [Information Projection and Approximate Inference for Structured Sparse Variables](https://mlsanity.com/paper/1607.03204/) (2016)

## Built on by (most cited first, 4 of 4)

- [Towards hyperparameter-free optimization with differential privacy](https://mlsanity.com/paper/2503.00703/) (2025)
- [Accelerating Differentially Private Federated Learning via Adaptive Extrapolation](https://mlsanity.com/paper/2504.09850/) (2025)
- [Budget Allocation for Unknown Value Functions in a Lipschitz Space](https://mlsanity.com/paper/2510.10605/) (2025)
- [INO-SGD: Addressing Utility Imbalance under Individualized Differential Privacy](https://mlsanity.com/paper/2605.07930/) (2026)

## Abstract (arXiv)

Hyperparameter optimization, also known as hyperparameter tuning, is a widely recognized technique for improving model performance. Regrettably, when training private ML models, many practitioners often overlook the privacy risks associated with hyperparameter optimization, which could potentially expose sensitive information about the underlying dataset. Currently, the sole existing approach to allow privacy-preserving hyperparameter optimization is to uniformly and randomly select hyperparameters for a number of runs, subsequently reporting the best-performing hyperparameter. In contrast, in non-private settings, practitioners commonly utilize ``adaptive'' hyperparameter optimization methods such as Gaussian process-based optimization, which select the next candidate based on information gathered from previous outputs. This substantial contrast between private and non-private hyperparameter optimization underscores a critical concern. In our paper, we introduce DP-HyPO, a pioneering framework for ``adaptive'' private hyperparameter optimization, aiming to bridge the gap between private and non-private hyperparameter optimization. To accomplish this, we provide a comprehensive differential privacy analysis of our framework. Furthermore, we empirically demonstrate the effectiveness of DP-HyPO on a diverse set of real-world datasets.