A View on Out-of-Distribution Identification from a Statistical Testing Theory Perspective
Not on the map. Why?
Abstract (arXiv)
We study the problem of efficiently detecting Out-of-Distribution (OOD) samples at test time in supervised and unsupervised learning contexts. While ML models are typically trained under the assumption that training and test data stem from the same distribution, this is often not the case in realistic settings, thus reliably detecting distribution shifts is crucial at deployment. We re-formulate the OOD problem under the lenses of statistical testing and then discuss conditions that render the OOD problem identifiable in statistical terms. Building on this framework, we study convergence guarantees of an OOD test based on the Wasserstein distance, and provide a simple empirical evaluation.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 56 references that are in the map. First posted on arXiv 5 May 2024.
Builds on
34 of 56 references.
References · 34
- 2017Attention Is All You Needcited by 59,932
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
- 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
- 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
- 2020Energy-based Out-of-distribution Detectioncited by 1,296
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
- 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
- 2017Deep Bayesian Active Learning with Image Datacited by 812
- 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
- 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
- 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
- 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
22 more
- 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
- 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
- 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
- 2021Unsolved Problems in ML Safetycited by 291
- 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
- 2018Statistical Aspects of Wasserstein Distancescited by 202
- 2015On Wasserstein Two Sample Testing and Related Families of Nonparametric Testscited by 165
- 2020Uncertainty Quantification and Deep Ensemblescited by 138
- 2023Modern Bayesian Experimental Designcited by 121
- 2022Is Out-of-Distribution Detection Learnable?cited by 109
- 2021Towards a Theoretical Framework of Out-of-Distribution Generalizationcited by 94
- 2021Understanding Failures in Out-of-Distribution Detection with Deep Generative Modelscited by 89
- 2021Provable Guarantees for Understanding Out-of-distribution Detectioncited by 78
- 2020The Case for Bayesian Deep Learningcited by 68
- 2021On the Impact of Spurious Correlation for Out-of-distribution Detectioncited by 68
- 2020An Evaluation of Change Point Detection Algorithmscited by 62
- 2019Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detectioncited by 35
- 2019From GAN to WGANcited by 34
- 2021A statistical framework for efficient out of distribution detection in deep neural networkscited by 27
- 2021Smooth p-Wasserstein Distance: Structure, Empirical Approximation, and Statistical Applicationscited by 15
- 2020Multivariate goodness-of-Fit tests based on Wasserstein distancecited by 11
- 2020Fast multivariate empirical cumulative distribution function with connection to kernel density estimationcited by 5
Built on by
Citing papers per year: 2024: 1.
1 paper.