mlsanityData to 14 Oct 2026

Uncertainty Quantification Metrics for Deep Regression

arXiv 2405.04278 ↗First posted 7 May 2024cs.LG cs.ROcited by 16

Not on the map. Why?

Abstract (arXiv)

When deploying deep neural networks on robots or other physical systems, the learned model should reliably quantify predictive uncertainty. A reliable uncertainty allows downstream modules to reason about the safety of its actions. In this work, we address metrics for evaluating such an uncertainty. Specifically, we focus on regression tasks, and investigate Area Under Sparsification Error (AUSE), Calibration Error, Spearman's Rank Correlation, and Negative Log-Likelihood (NLL). Using synthetic regression datasets, we look into how those metrics behave under four typical types of uncertainty, their stability regarding the size of the test set, and reveal their strengths and weaknesses. Our results indicate that Calibration Error is the most stable and interpretable metric, but AUSE and NLL also have their respective use cases. We discourage the usage of Spearman's Rank Correlation for evaluating uncertainties and recommend replacing it with AUSE.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 24 of its 39 references that are in the map. First posted on arXiv 7 May 2024.

Builds on

24 of 39 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  4. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  5. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  6. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 881
  7. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
  8. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 422
  9. 2021Revisiting the Calibration of Modern Neural Networkscited by 390
  10. 2019Learning Loss for Active Learningcited by 336
  11. 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 305
  12. 2018Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flowcited by 135
12 more
  1. 2019A General Framework for Uncertainty Estimation in Deep Learningcited by 119
  2. 2018Uncertainty in Neural Networks: Approximately Bayesian Ensemblingcited by 94
  3. 2020Uncertainty-Aware CNNs for Depth Completion: Uncertainty from Beginning to Endcited by 55
  4. 2022A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awarenesscited by 49
  5. 2020Wasserstein Distances for Stereo Disparity Estimationcited by 38
  6. 2023Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensemblescited by 30
  7. 2017ProbFlow: Joint Optical Flow and Uncertainty Estimationcited by 24
  8. 2019Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernelcited by 19
  9. 2020Estimating Uncertainty in Neural Networks for Cardiac MRI Segmentation: A Benchmark Studycited by 19
  10. 2021NOMU: Neural Optimization-based Model Uncertaintycited by 16
  11. 2023Learning Sample Difficulty from Pre-trained Models for Reliable Predictioncited by 15
  12. 2019Energy-Based Models for Deep Probabilistic Regressioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 8.

16 papers.

Sort:

  1. 2025Continuous Normalizing Flows for Uncertainty-Aware Human Pose Estimationcited by 2
  2. 2025CLUE: Neural Networks Calibration via Learning Uncertainty-Error alignmentcited by 2
  3. 2025Probabilistic Neural Networks (PNNs) with t-Distributed Outputs: Adaptive Prediction Intervals Beyond Gaussian Assumptionscited by 1
  4. 2025Evaluating the Quality of the Quantified Uncertainty for (Re)Calibration of Data-Driven Regression Modelscited by 1
  5. 2026Principled Confidence Estimation for Deep Computed Tomographycited by 1
  6. 2026Laplace Approximation for Bayesian Tensor Network Kernel Machinescited by 1
  7. 2025TRUST: Test-time Resource Utilization for Superior Trustworthinesscited by 0
  8. 2025MATTER: Multiscale Attention for Registration Error Regressioncited by 0
  9. 2025ViscNet: Vision-Based In-line Viscometry for Fluid Mixing Processcited by 0
  10. 2025R^2-HGP: A Double-Regularized Gaussian Process for Heterogeneous Transfer Learningcited by 0
  11. 2026GeoBlur: Epipolar Geometry Estimation from a Single Motion-Blurred Imagecited by 0
  12. 2026Neuromorphic Monocular Depth Estimation with Uncertainty Modelingcited by 0
4 more
  1. 2026Uncertainty-Calibrated Recommendations for Low-Active Userscited by 0
  2. 2026Error Aware Distribution Prediction for Lightweight Implicit Neural Representationscited by 0
  3. 2026Gaussian-Mixture Latent Flow for Stochastic 3D Human Motion Predictioncited by 0
  4. 2026A Calibration Audit of Confidence in Feed-Forward 3D Reconstruction Modelscited by 0