mlsanityData to 7 Oct 2026

Uncertainty Quantification with the Empirical Neural Tangent Kernel

arXiv 2502.02870 ↗First posted 5 Feb 2025stat.ML cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

While neural networks have demonstrated impressive performance across various tasks, accurately quantifying uncertainty in their predictions is essential to ensure their trustworthiness and enable widespread adoption in critical systems. Several Bayesian uncertainty quantification (UQ) methods exist that are either cheap or reliable, but not both. We propose a post-hoc, sampling-based UQ method for over-parameterized networks at the end of training. Our approach constructs efficient and meaningful deep ensembles by employing a (stochastic) gradient-descent sampling process on appropriately linearized networks. We demonstrate that our method effectively approximates the posterior of a Gaussian process using the empirical Neural Tangent Kernel. Through a series of numerical experiments, we show that our method not only outperforms competing approaches in computational efficiency-often reducing costs by multiple factors-but also maintains state-of-the-art performance across a variety of UQ metrics for both regression and classification tasks.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 55 of its 89 references that are in the map. First posted on arXiv 5 Feb 2025.

Builds on

55 of 89 references.

References · 55

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  7. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  8. 2016YOLO9000: Better, Faster, Strongercited by 1,429
  9. 2019Natural Adversarial Examplescited by 1,401
  10. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  11. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  12. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
38 more
  1. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  2. 2014Convex Optimization: Algorithms and Complexitycited by 569
  3. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  4. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 526
  5. 2019Measuring Calibration in Deep Learningcited by 443
  6. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  7. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  8. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  9. 2020Loss landscapes and optimization in over-parameterized non-linear systems and neural networkscited by 231
  10. 2021Deep Deterministic Uncertainty: A Simple Baselinecited by 200
  11. 2020Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernelcited by 185
  12. 2020Feature Learning in Infinite-Width Neural Networkscited by 182
  13. 2020Training independent subnetworks for robust predictioncited by 177
  14. 2023Handbook of Convergence Theorems for (Stochastic) Gradient Methodscited by 158
  15. 2023In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluationcited by 132
  16. 2019Adversarial Examples Are a Natural Consequence of Test Error in Noisecited by 114
  17. 2020Bayesian Deep Ensembles via the Neural Tangent Kernelcited by 90
  18. 2019Approximate Inference Turns Deep Networks into Gaussian Processescited by 89
  19. 2019'In-Between' Uncertainty in Bayesian Neural Networkscited by 79
  20. 2019Adaptive Gradient Descent without Descentcited by 79
  21. 2022Deep Ensembles Work, But Are They Necessary?cited by 63
  22. 2021DOCTOR: A Simple Method for Detecting Misclassification Errorscited by 52
  23. 2024Estimating Epistemic and Aleatoric Uncertainty with a Single Modelcited by 52
  24. 2024The Elements of Differentiable Programmingcited by 41
  25. 2022Nonparametric Uncertainty Quantification for Single Deterministic Neural Networkcited by 36
  26. 2021Is MC Dropout Bayesian?cited by 35
  27. 2018On exponential convergence of SGD in non-convex over-parametrized learningcited by 33
  28. 2021Deep Ensembles from a Bayesian Perspectivecited by 27
  29. 2022Accelerated Linearized Laplace Approximation for Bayesian Deep Learningcited by 27
  30. 2023Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorialcited by 27
  31. 2021Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learningcited by 25
  32. 2022Sampling-based inference for large linear models, with application to linearised Laplacecited by 18
  33. 2020Encoding the latent posterior of Bayesian Neural Networks for uncertainty quantificationcited by 17
  34. 2023Variational Linearized Laplace Approximation for Bayesian Deep Learningcited by 17
  35. 2022On the Power-Law Hessian Spectrums in Deep Learningcited by 14
  36. 2023A Data-Driven Measure of Relative Uncertainty for Misclassification Detectioncited by 14
  37. 2023The Interpolating Information Criterion for Overparameterized Modelscited by 11
  38. 2023Efficient Uncertainty Quantification and Reduction for Over-Parameterized Neural Networkscited by 10

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 6.

11 papers.

  1. 2025Models of Heavy-Tailed Mechanistic Universalitycited by 9
  2. 2025Determinant Estimation under Memory Constraints and Neural Scaling Lawscited by 3
  3. 2025Contextual Similarity Distillation: Ensemble Uncertainties with a Single Modelcited by 3
  4. 2025Finite-Width Neural Tangent Kernels from Feynman Diagramscited by 3
  5. 2026Richer Bayesian Last Layers with Subsampled NTK Featurescited by 3
  6. 2026Efficient Analysis of the Distilled Neural Tangent Kernelcited by 1
  7. 2026Is the Last Layer Sufficient for Uncertainty Quantification?cited by 1
  8. 2026Scalable Uncertainty Quantification for Extreme Weather Forecasting via Empirical Neural Tangent Kernelscited by 1
  9. 2025Spectral Estimation with Free Decompressioncited by 0
  10. 2026On the Equivalence of Random Network Distillation, Deep Ensembles, and Bayesian Inferencecited by 0
  11. 2026AdaptNTK: Adaptive Uncertainty Quantification and Active Learning for Neural Network Potentialscited by 0