Beyond the Norms: Detecting Prediction Errors in Regression Models
Not on the map. Why?
Abstract (arXiv)
This paper tackles the challenge of detecting unreliable behavior in regression algorithms, which may arise from intrinsic variability (e.g., aleatoric uncertainty) or modeling errors (e.g., model uncertainty). First, we formally introduce the notion of unreliability in regression, i.e., when the output of the regressor exceeds a specified discrepancy (or error). Then, using powerful tools for probabilistic modeling, we estimate the discrepancy density, and we measure its statistical diversity using our proposed metric for statistical dissimilarity. In turn, this allows us to derive a data-driven score that expresses the uncertainty of the regression outcome. We show empirical improvements in error detection for multiple regression tasks, consistently outperforming popular baseline approaches, and contributing to the broader field of uncertainty quantification and safe machine learning systems. Our code is available at https://zenodo.org/records/11281964.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 27 of its 44 references that are in the map. First posted on arXiv 11 Jun 2024.
Builds on
27 of 44 references.
References · 27
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
- 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
- 2016Concrete Problems in AI Safetycited by 1,582
- 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
- 2019Conformalized Quantile Regressioncited by 620
- 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 417
- 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
- 2019On instabilities of deep learning in image reconstruction - Does AI come at a cost?cited by 211
15 more
- 2021Trustworthy AI: From Principles to Practicescited by 206
- 2019Addressing Failure Prediction by Learning Model Confidencecited by 183
- 2020Self-Adaptive Training: beyond Empirical Risk Minimizationcited by 126
- 2018High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approachcited by 117
- 2020Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantificationcited by 77
- 2021DOCTOR: A Simple Method for Detecting Misclassification Errorscited by 52
- 2020Individual Calibration with Randomized Forecastingcited by 50
- 2022Nonparametric Uncertainty Quantification for Single Deterministic Neural Networkcited by 36
- 2021Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networkscited by 32
- 2020Regression with reject option and application to kNNcited by 30
- 2017Fast calibrated additive quantile regressioncited by 26
- 2022A Differential Entropy Estimator for Training Neural Networkscited by 20
- 2019Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernelcited by 19
- 2021Adversarial Robustness via Fisher-Rao Regularizationcited by 19
- 2023A Data-Driven Measure of Relative Uncertainty for Misclassification Detectioncited by 14
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.