mlsanityData to 7 Oct 2026

A prediction rigidity formalism for low-cost uncertainties in trained neural networks

arXiv 2403.02251 ↗First posted 4 Mar 2024stat.ML cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Regression methods are fundamental for scientific and technological applications. However, fitted models can be highly unreliable outside of their training domain, and hence the quantification of their uncertainty is crucial in many of their applications. Based on the solution of a constrained optimization problem, we propose "prediction rigidities" as a method to obtain uncertainties of arbitrary pre-trained regressors. We establish a strong connection between our framework and Bayesian inference, and we develop a last-layer approximation that allows the new method to be applied to neural networks. This extension affords cheap uncertainties without any modification to the neural network itself or its training procedure. We show the effectiveness of our method on a wide range of regression tasks, ranging from simple toy models to applications in chemistry and meteorology.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 30 of its 66 references that are in the map. First posted on arXiv 4 Mar 2024.

Builds on

30 of 66 references.

References · 30

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  4. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  5. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  6. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  7. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  8. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  9. 2017Deep Neural Networks as Gaussian Processescited by 617
  10. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  11. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
  12. 2015Deep Kernel Learningcited by 440
18 more
  1. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  2. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  3. 2017Snapshot Ensembles: Train 1, get M for freecited by 349
  4. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  5. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
  6. 2020Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learningcited by 208
  7. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  8. 2015Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)cited by 175
  9. 2016Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivitycited by 138
  10. 2020Uncertainty Quantification and Deep Ensemblescited by 138
  11. 2020Bayesian Deep Ensembles via the Neural Tangent Kernelcited by 90
  12. 2019Approximate Inference Turns Deep Networks into Gaussian Processescited by 89
  13. 2021The Promises and Pitfalls of Deep Kernel Learningcited by 81
  14. 2022A Framework and Benchmark for Deep Batch Active Learning for Regressioncited by 47
  15. 2020On the infinite width limit of neural networks with a standard parameterizationcited by 38
  16. 2021Deep Ensembles from a Bayesian Perspectivecited by 27
  17. 2022Wide and Deep Neural Networks Achieve Optimality for Classificationcited by 12
  18. 2023Fusion framework and multimodality for the Laplacian approximation of Bayesian neural networkscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025PET-MAD, a lightweight universal interatomic potential for advanced materials modelingcited by 23
  2. 2025FlashMD: long-stride, universal prediction of molecular dynamicscited by 11
  3. 2025Enhancing Machine Learning Potentials through Transfer Learning across Chemical Elementscited by 2
  4. 2026Knowing when to trust machine-learned interatomic potentialscited by 1
  5. 2025Comparing the latent features of universal machine-learning interatomic potentialscited by 0
  6. 2026Full-data accuracy with fewer labels for training and fine-tuning machine-learning force fieldscited by 0
  7. 2026AdaptNTK: Adaptive Uncertainty Quantification and Active Learning for Neural Network Potentialscited by 0