mlsanityData to 7 Oct 2026

Structure and Distribution Metric for Quantifying the Quality of Uncertainty: Assessing Gaussian Processes, Deep Neural Nets, and Deep Neural Operators for Regression

arXiv 2203.04515 ↗First posted 9 Mar 2022cs.LG stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

We propose two bounded comparison metrics that may be implemented to arbitrary dimensions in regression tasks. One quantifies the structure of uncertainty and the other quantifies the distribution of uncertainty. The structure metric assesses the similarity in shape and location of uncertainty with the true error, while the distribution metric quantifies the supported magnitudes between the two. We apply these metrics to Gaussian Processes (GPs), Ensemble Deep Neural Nets (DNNs), and Ensemble Deep Neural Operators (DNOs) on high-dimensional and nonlinear test cases. We find that comparing a model's uncertainty estimates with the model's squared error provides a compelling ground truth assessment. We also observe that both DNNs and DNOs, especially when compared to GPs, provide encouraging metric values in high dimensions with either sparse or plentiful data.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 12 of its 22 references that are in the map. First posted on arXiv 9 Mar 2022.

Builds on

12 of 22 references.

References · 12

  1. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  3. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  4. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  5. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  6. 2017Snapshot Ensembles: Train 1, get M for freecited by 349
  7. 2019Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Visioncited by 182
  8. 2022Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisonscited by 169
  9. 2018High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approachcited by 117
  10. 2019Quality of Uncertainty Quantification for Bayesian Neural Network Inferencecited by 58
  11. 2018A sequential sampling strategy for extreme event statistics in nonlinear dynamical systemscited by 14
  12. 2018Tight Prediction Intervals Using Expanded Interval Minimizationcited by 4

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 2.

5 papers.

  1. 2022NeuralUQ: A comprehensive library for uncertainty quantification in neural differential equations and operatorscited by 33
  2. 2022Discovering and forecasting extreme events via active learning in neural operatorscited by 32
  3. 2023Improved generalization with deep neural operators for engineering systems: Path towards digital twincited by 19
  4. 2023How Reliable is Your Regression Model's Uncertainty Under Real-World Distribution Shifts?cited by 14
  5. 2022Information FOMO: The unhealthy fear of missing out on information. A method for removing misleading data for healthier modelscited by 1