mlsanityData to 14 Oct 2026

On Uncertainty Quantification for Near-Bayes Optimal Algorithms

arXiv 2403.19381 ↗First posted 28 Mar 2024stat.ML cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Bayesian modelling allows for the quantification of predictive uncertainty which is crucial in safety-critical applications. Yet for many machine learning (ML) algorithms, it is difficult to construct or implement their Bayesian counterpart. In this work we present a promising approach to address this challenge, based on the hypothesis that commonly used ML algorithms are efficient across a wide variety of tasks and may thus be near Bayes-optimal w.r.t. an unknown task distribution. We prove that it is possible to recover the Bayesian posterior defined by the task distribution, which is unknown but optimal in this setting, by building a martingale posterior using the algorithm. We further propose a practical uncertainty quantification method that apply to general ML algorithms. Experiments based on a variety of non-NN and NN algorithms demonstrate the efficacy of our method.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 62 of its 119 references that are in the map. First posted on arXiv 28 Mar 2024.

Builds on

62 of 119 references.

References · 62

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  6. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  7. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  9. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  10. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
  11. 2018On Lazy Training in Differentiable Programmingcited by 675
  12. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
38 more
  1. 2021Tabular Data: Deep Learning is Not All You Needcited by 535
  2. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 507
  3. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 443
  4. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 383
  5. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 371
  6. 2021Transformers Can Do Bayesian Inferencecited by 360
  7. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 308
  8. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 219
  9. 2020Few-Shot Learning via Learning the Representation, Provablycited by 214
  10. 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
  11. 2019Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Visioncited by 182
  12. 2019Dice Loss for Data-imbalanced NLP Taskscited by 182
  13. 2017OpenML Benchmarking Suitescited by 178
  14. 2020Stability of Stochastic Gradient Descent on Nonsmooth Convex Lossescited by 173
  15. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  16. 2020Provable Meta-Learning of Linear Representationscited by 161
  17. 2022If Influence Functions are the Answer, Then What is the Question?cited by 144
  18. 2017Doubly Stochastic Variational Inference for Deep Gaussian Processescited by 143
  19. 2019Functional Variational Bayesian Neural Networkscited by 127
  20. 2021Bayesian Neural Network Priors Revisitedcited by 121
  21. 2020Implicit Regularization in Deep Learning May Not Be Explainable by Normscited by 119
  22. 2023Bayesian Low-rank Adaptation for Large Language Modelscited by 115
  23. 2021Repulsive Deep Ensembles are Bayesiancited by 104
  24. 2020Causal Autoregressive Flowscited by 99
  25. 2018Uncertainty in Neural Networks: Approximately Bayesian Ensemblingcited by 94
  26. 2020Bayesian Deep Ensembles via the Neural Tangent Kernelcited by 90
  27. 2020PACOH: Bayes-Optimal Meta-Learning with PAC-Guaranteescited by 85
  28. 2013A PAC-Bayesian bound for Lifelong Learningcited by 69
  29. 2019Rates of Convergence for Sparse Variational Gaussian Process Regressioncited by 63
  30. 2020AutoML to Date and Beyond: Challenges and Opportunitiescited by 63
  31. 2020All You Need is a Good Functional Prior for Bayesian Deep Learningcited by 61
  32. 2020A statistical theory of cold posteriors in deep neural networkscited by 54
  33. 2020Bayesian Workflowcited by 49
  34. 2022On Uncertainty, Tempering, and Data Augmentation in Bayesian Classificationcited by 46
  35. 2021Martingale posterior distributionscited by 45
  36. 2020Sparse Gaussian Processes with Spherical Harmonic Featurescited by 43
  37. 2018Variational Implicit Processescited by 41
  38. 2019Function Space Particle Optimization for Bayesian Neural Networkscited by 41

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2025C-LoRA: Contextual Low-Rank Adaptation for Uncertainty Estimation in Large Language Modelscited by 9
  2. 2024Quasi-Bayes meets Vinescited by 5
  3. 2026Variational predictive resamplingcited by 0