# The Test of Tests: A Framework For Differentially Private Hypothesis Testing

arXiv: https://arxiv.org/abs/2302.04260 · HTML: https://mlsanity.com/paper/2302.04260/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 44 references that are in the map.
- First posted on arXiv 8 Feb 2023.

## Builds on (most cited first, 15 of 15)

- [Differentially Private Ordinary Least Squares](https://mlsanity.com/paper/1507.02482/) (2015)
- [The Structure of Optimal Private Tests for Simple Hypotheses](https://mlsanity.com/paper/1811.11148/) (2018)
- [Differentially Private Chi-Squared Hypothesis Testing: Goodness of Fit and Independence Testing](https://mlsanity.com/paper/1602.03090/) (2016)
- [Privacy-preserving parametric inference: a case for robust statistics](https://mlsanity.com/paper/1911.10167/) (2019)
- [Private Identity Testing for High-Dimensional Distributions](https://mlsanity.com/paper/1905.11947/) (2019)
- [Differentially private significance tests for regression coefficients](https://mlsanity.com/paper/1705.09561/) (2017)
- [Differentially Private Nonparametric Hypothesis Testing](https://mlsanity.com/paper/1903.09364/) (2019)
- [Private High-Dimensional Hypothesis Testing](https://mlsanity.com/paper/2203.01537/) (2022)
- [Priv'IT: Private and Sample Efficient Identity Testing](https://mlsanity.com/paper/1703.10127/) (2017)
- [Improved Differentially Private Analysis of Variance](https://mlsanity.com/paper/1903.00534/) (2019)
- [Differentially Private Hypothesis Testing with the Subsampled and Aggregated Randomized Response Mechanism](https://mlsanity.com/paper/2208.06803/) (2022)
- [Hypothesis Testing for Differentially Private Linear Regression](https://mlsanity.com/paper/2206.14449/) (2022)
- [Comparing Population Means under Local Differential Privacy: with Significance and Power](https://mlsanity.com/paper/1803.09027/) (2018)
- [Differentially Private ANOVA Testing](https://mlsanity.com/paper/1711.01335/) (2017)
- [Differentially Private Regression for Discrete-Time Survival Analysis](https://mlsanity.com/paper/1708.07436/) (2017)

## Built on by (most cited first, 9 of 9)

- [Robust Kernel Hypothesis Testing under Data Corruption](https://mlsanity.com/paper/2405.19912/) (2024)
- [Differentially private Bayesian tests](https://mlsanity.com/paper/2401.15502/) (2024)
- [Evasive Active Hypothesis Testing with Deep Neuroevolution: The Single- and Multi-Agent Cases](https://mlsanity.com/paper/2403.10112/) (2024)
- [Fiducial Matching: Differentially Private Inference for Categorical Data](https://mlsanity.com/paper/2507.11762/) (2025)
- [DP-SPRT: Differentially Private Sequential Probability Ratio Tests](https://mlsanity.com/paper/2508.06377/) (2025)
- [Differentially Private E-Values](https://mlsanity.com/paper/2510.18654/) (2025)
- [Optimal Rates for Differentially Private Hypothesis Testing with E-values](https://mlsanity.com/paper/2605.28952/) (2026)
- [Differentially Private Conditional Independence Testing](https://mlsanity.com/paper/2306.06721/) (2023)
- [Near-Optimal Private Tests for Simple and MLR Hypotheses](https://mlsanity.com/paper/2601.21959/) (2026)

## Abstract (arXiv)

We present a generic framework for creating differentially private versions of any hypothesis test in a black-box way. We analyze the resulting tests analytically and experimentally. Most crucially, we show good practical performance for small data sets, showing that at epsilon = 1 we only need 5-6 times as much data as in the fully public setting. We compare our work to the one existing framework of this type, as well as to several individually-designed private hypothesis tests. Our framework is higher power than other generic solutions and at least competitive with (and often better than) individually-designed tests.