# On the Robustness of Kernel Goodness-of-Fit Tests

arXiv: https://arxiv.org/abs/2408.05854 · HTML: https://mlsanity.com/paper/2408.05854/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 111 references that are in the map.
- First posted on arXiv 11 Aug 2024.

## Builds on (most cited first, 30 of 44)

- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [Measuring Sample Quality with Kernels](https://mlsanity.com/paper/1703.01717/) (2017)
- [Measuring Sample Quality with Stein's Method](https://mlsanity.com/paper/1506.03039/) (2015)
- [MMD Aggregated Two-Sample Test](https://mlsanity.com/paper/2110.15073/) (2021)
- [Control functionals for Monte Carlo integration](https://mlsanity.com/paper/1410.2392/) (2014)
- [Minimum Stein Discrepancy Estimators](https://mlsanity.com/paper/1906.08283/) (2019)
- [Robust Generalised Bayesian Inference for Intractable Likelihoods](https://mlsanity.com/paper/2104.07359/) (2021)
- [Statistical Inference for Generative Models with Maximum Mean Discrepancy](https://mlsanity.com/paper/1906.05944/) (2019)
- [Maximum Mean Discrepancy Test is Aware of Adversarial Attacks](https://mlsanity.com/paper/2010.11415/) (2020)
- [A Linear-Time Kernel Goodness-of-Fit Test](https://mlsanity.com/paper/1705.07673/) (2017)
- [Robust Bayesian Inference for Simulator-based Models via the MMD Posterior Bootstrap](https://mlsanity.com/paper/2202.04744/) (2022)
- [MMD-Bayes: Robust Bayesian Estimation via Maximum Mean Discrepancy](https://mlsanity.com/paper/1909.13339/) (2019)
- [Blindness of score-based methods to isolated components and mixing proportions](https://mlsanity.com/paper/2008.10087/) (2020)
- [Stein's Method Meets Computational Statistics: A Review of Some Recent Developments](https://mlsanity.com/paper/2105.03481/) (2021)
- [A Wild Bootstrap for Degenerate Kernel Tests](https://mlsanity.com/paper/1408.5404/) (2014)
- [Robust Hypothesis Testing Using Wasserstein Uncertainty Sets](https://mlsanity.com/paper/1805.10611/) (2018)
- [Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence](https://mlsanity.com/paper/1912.05737/) (2019)
- [Random Feature Stein Discrepancies](https://mlsanity.com/paper/1806.07788/) (2018)
- [Nystr"om Kernel Mean Embeddings](https://mlsanity.com/paper/2201.13055/) (2022)
- [A Finite-Particle Convergence Rate for Stein Variational Gradient Descent](https://mlsanity.com/paper/2211.09721/) (2022)
- [KSD Aggregated Goodness-of-fit Test](https://mlsanity.com/paper/2202.00824/) (2022)
- [Outlier-robust Kalman Filtering through Generalised Bayes](https://mlsanity.com/paper/2405.05646/) (2024)
- [Spectral Regularized Kernel Two-Sample Tests](https://mlsanity.com/paper/2212.09201/) (2022)
- [On the Optimality of Kernel-Embedding Based Goodness-of-Fit Tests](https://mlsanity.com/paper/1709.08148/) (2017)
- [Composite Goodness-of-fit Tests with Kernels](https://mlsanity.com/paper/2111.10275/) (2021)
- [Robust and Scalable Bayesian Online Changepoint Detection](https://mlsanity.com/paper/2302.04759/) (2023)
- [Targeted Separation and Convergence with Kernel Discrepancies](https://mlsanity.com/paper/2209.12835/) (2022)
- [Robust and Conjugate Gaussian Process Regression](https://mlsanity.com/paper/2311.00463/) (2023)
- [A Stein Goodness-of-fit Test for Directional Distributions](https://mlsanity.com/paper/2002.06843/) (2020)

## Built on by (most cited first, 7 of 7)

- [Conformalized Credal Regions for Classification with Ambiguous Ground Truth](https://mlsanity.com/paper/2411.04852/) (2024)
- [Robust and Conjugate Spatio-Temporal Gaussian Processes](https://mlsanity.com/paper/2502.02450/) (2025)
- [Conformal Prediction Regions are Imprecise Highest Density Regions](https://mlsanity.com/paper/2502.06331/) (2025)
- [Multi-Output Robust and Conjugate Gaussian Processes](https://mlsanity.com/paper/2510.26401/) (2025)
- [Kernel Tests of Equivalence](https://mlsanity.com/paper/2603.10886/) (2026)
- [Minimax Lower Bounds of Kernel Discrepancy Estimation: MMD, HSIC, KSD](https://mlsanity.com/paper/2607.24235/) (2026)
- [Optimal Transport for ε-Contaminated Credal Sets: To the Memory of Sayan Mukherjee](https://mlsanity.com/paper/2410.03267/) (2024)

## Abstract (arXiv)

Goodness-of-fit testing is often criticized for its lack of practical relevance: since ``all models are wrong'', the null hypothesis that the data conform to our model is ultimately always rejected as the sample size grows. Despite this, probabilistic models are still used extensively, raising the more pertinent question of whether the model is good enough for the task at hand. This question can be formalized as a robust goodness-of-fit testing problem by asking whether the data were generated from a distribution that is a mild perturbation of the model. In this paper, we show that existing kernel goodness-of-fit tests are not robust under common notions of robustness including both qualitative and quantitative robustness. We further show that robustification techniques using tilted kernels, while effective in the parameter estimation literature, are not sufficient to ensure both types of robustness in the testing setting. To address this, we propose the first robust kernel goodness-of-fit test, which resolves this open problem by using kernel Stein discrepancy (KSD) balls. This framework encompasses many well-known perturbation models, such as Huber's contamination and density-band models.