mlsanityData to 7 Oct 2026

A framework for benchmarking uncertainty in deep regression

arXiv 2109.09048 ↗First posted 10 Sep 2021cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

We propose a framework for the assessment of uncertainty quantification in deep regression. The framework is based on regression problems where the regression function is a linear combination of nonlinear functions. Basically, any level of complexity can be realized through the choice of the nonlinear functions and the dimensionality of their domain. Results of an uncertainty quantification for deep regression are compared against those obtained by a statistical reference method. The reference method utilizes knowledge of the underlying nonlinear functions and is based on a Bayesian linear regression using a reference prior. Reliability of uncertainty quantification is assessed in terms of coverage probabilities, and accuracy through the size of calculated uncertainties. We illustrate the proposed framework by applying it to current approaches for uncertainty quantification in deep regression. The flexibility, together with the availability of a reference solution, makes the framework suitable for defining benchmark sets for uncertainty quantification.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 23 of its 64 references that are in the map. First posted on arXiv 10 Sep 2021.

Builds on

23 of 64 references.

References · 23

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  4. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  5. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  6. 2015Weight Uncertainty in Neural Networkscited by 554
  7. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  8. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 417
  9. 2017Snapshot Ensembles: Train 1, get M for freecited by 349
  10. 2017Concrete Dropoutcited by 227
  11. 2019Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Visioncited by 182
  12. 2018An overview of deep learning in medical imaging focusing on MRIcited by 140
11 more
  1. 2018Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flowcited by 135
  2. 2018Uncertainty in Neural Networks: Approximately Bayesian Ensemblingcited by 94
  3. 2020Bayesian Deep Ensembles via the Neural Tangent Kernelcited by 90
  4. 2019Evaluating and Calibrating Uncertainty Prediction in Regression Taskscited by 85
  5. 2021Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learningcited by 81
  6. 2019Quality of Uncertainty Quantification for Bayesian Neural Network Inferencecited by 58
  7. 2020Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologiescited by 41
  8. 2020Deeply Uncertain: Comparing Methods of Uncertainty Quantification in Deep Learning Algorithmscited by 34
  9. 2021Deep Ensembles from a Bayesian Perspectivecited by 27
  10. 2020Beyond Marginal Uncertainty: How Accurately can Bayesian Regression Models Estimate Posterior Predictive Correlations?cited by 21
  11. 2020Deep Neural Networks for Computational Optical Form Measurementscited by 2

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 2.

4 papers.

  1. 2022Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisonscited by 169
  2. 2023MUBen: Benchmarking the Uncertainty of Molecular Representation Modelscited by 11
  3. 2024Lightning UQ Box: A Comprehensive Framework for Uncertainty Quantification in Deep Learningcited by 3
  4. 2024Measurement Uncertainty: Relating the uncertainties of physical and virtual measurementscited by 0