Scalable Bayesian Monte Carlo: fast uncertainty estimation beyond deep ensembles
Not on the map. Why?
Abstract (arXiv)
This work introduces a new method designed for Bayesian deep learning called scalable Bayesian Monte Carlo (SBMC). The method is comprised of a model and an algorithm. The model interpolates between a point estimator and the posterior. The algorithm is a parallel implementation of sequential Monte Carlo sampler (SMC_parallel) or Markov chain Monte Carlo (MCMC_parallel). We collectively refer to these consistent (asymptotically unbiased) algorithms as Bayesian Monte Carlo (BMC), and any such algorithm can be used in our SBMC method. The utility of the method is demonstrated on practical examples: MNIST, CIFAR, IMDb. A systematic numerical study reveals that for the same wall-clock time as state-of-the-art (SOTA) methods like deep ensembles (DE), SBMC achieves comparable or better accuracy and substantially improved uncertainty quantification (UQ)--in particular, epistemic UQ. This is demonstrated on the downstream task of estimating the confidence in predictions, which can be used for reliability assessment or abstention decisions.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 40 of its 90 references that are in the map. First posted on arXiv 19 May 2025.
Builds on
40 of 90 references.
References · 40
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
- 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
- 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
- 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
- 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
- 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
28 more
- 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
- 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
- 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
- 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 365
- 2014Stochastic Gradient Hamiltonian Monte Carlocited by 343
- 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 308
- 2024Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoderscited by 224
- 2019Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Visioncited by 182
- 2021LogME: Practical Assessment of Pre-trained Models for Transfer Learningcited by 172
- 2024Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraphcited by 105
- 2018Uncertainty in Neural Networks: Approximately Bayesian Ensemblingcited by 94
- 2025OTTER: A Vision-Language-Action Model with Text-Aware Visual Feature Extractioncited by 54
- 2020Aleatoric and Epistemic Uncertainty with Random Forestscited by 43
- 2020An invitation to sequential Monte Carlo samplerscited by 33
- 2016Patterns of Scalable Bayesian Inferencecited by 25
- 2021Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learningcited by 25
- 2014Firefly Monte Carlo: Exact MCMC with Subsets of Datacited by 23
- 2019Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernelcited by 19
- 2024Hallucination Detection in LLMs: Fast and Memory-Efficient Fine-Tuned Modelscited by 15
- 2024Scalable Bayesian Learning with posteriorscited by 13
- 2024Sampling from Bayesian Neural Network Posteriors with Symmetric Minibatch Splitting Langevin Dynamicscited by 13
- 2025Microcanonical Langevin Ensembles: Advancing the Sampling of Bayesian Neural Networkscited by 10
- 2021Nested hat R: Assessing the convergence of Markov chain Monte Carlo when running many short chainscited by 9
- 2025Probabilistic Artificial Intelligencecited by 6
- 2015Accelerated dimension-independent adaptive Metropoliscited by 4
- 2018Subsampling Sequential Monte Carlo for Static Bayesian Modelscited by 3
- 2024Evaluating Bayesian deep learning for radio galaxy classificationcited by 2
- 2024Comparison of parallel SMC and MCMC for Bayesian deep learningcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.