mlsanityData to 7 Oct 2026

Sampling from Bayesian Neural Network Posteriors with Symmetric Minibatch Splitting Langevin Dynamics

arXiv 2410.19780 ↗First posted 14 Oct 2024stat.ML cs.LG cs.NAcited by 12

Not on the map. Why?

Abstract (arXiv)

We propose a scalable kinetic Langevin dynamics algorithm for sampling parameter spaces of big data and AI applications. Our scheme combines a symmetric forward/backward sweep over minibatches with a symmetric discretization of Langevin dynamics. For a particular Langevin splitting method (UBU), we show that the resulting Symmetric Minibatch Splitting-UBU (SMS-UBU) integrator has bias O(h2 d1/2) in dimension d>0 with stepsize h>0, despite only using one minibatch per iteration, thus providing excellent control of the sampling bias as a function of the stepsize. We apply the algorithm to explore local modes of the posterior distribution of Bayesian neural networks (BNNs) and evaluate the calibration performance of the posterior predictive probabilities for neural networks with convolutional neural network architectures for classification problems on three different datasets (Fashion-MNIST, Celeb-A and chest X-ray). Our results indicate that BNNs sampled with SMS-UBU can offer significantly better calibration performance compared to standard methods of training and stochastic weight averaging.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 38 of its 76 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

38 of 76 references.

References · 38

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  2. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  3. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  4. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 627
  5. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  6. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  7. 2019Measuring Calibration in Deep Learningcited by 443
  8. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 341
  9. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  10. 2021The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networkscited by 285
  11. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  12. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
26 more
  1. 2017Underdamped Langevin MCMC: A non-asymptotic analysiscited by 131
  2. 2018Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adamcited by 131
  3. 2019Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networkscited by 127
  4. 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
  5. 2022Deep Ensembles Work, But Are They Necessary?cited by 63
  6. 2018On sampling from a log-concave density using kinetic Langevin diffusionscited by 61
  7. 2019Stochastic gradient Markov chain Monte Carlocited by 56
  8. 2020Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splittingcited by 54
  9. 2015On Markov chain Monte Carlo methods for tall datacited by 50
  10. 2022Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Taskscited by 49
  11. 2019Shaping the learning landscape in neural networks around wide flat minimacited by 47
  12. 2019Challenges in Markov chain Monte Carlo for Bayesian neural networkscited by 43
  13. 2015From Facial Parts Responses to Face Detection: A Deep Learning Approachcited by 42
  14. 2016On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integratorscited by 35
  15. 2023On Efficient Training of Large-Scale Deep Learning Models: A Literature Reviewcited by 26
  16. 2018On the Theory of Variance Reduction for Stochastic Gradient Monte Carlocited by 19
  17. 2021A Bayesian neural network predicts the dissolution of compact planetary systemscited by 15
  18. 2023Fortuna: A Library for Uncertainty Quantification in Deep Learningcited by 14
  19. 2023When does Metropolized Hamiltonian Monte Carlo provably outperform Metropolis-adjusted Langevin algorithm?cited by 12
  20. 2021Wasserstein distance estimates for the distributions of numerical approximations to ergodic stochastic differential equationscited by 8
  21. 2023Unbiased Kinetic Langevin Monte Carlo with Inexact Gradientscited by 8
  22. 2024Convergence of Unadjusted Langevin in High Dimensions: Delocalization of Biascited by 4
  23. 2021Revisiting the Effects of Stochasticity for Hamiltonian Samplerscited by 3
  24. 2023Reflection coupling for unadjusted generalized Hamiltonian Monte Carlo in the nonconvex stochastic gradient casecited by 3
  25. 2024Correction to "Wasserstein distance estimates for the distributions of numerical approximations to ergodic stochastic differential equations"cited by 2
  26. 2021Quantifying the mini-batching error in Bayesian inference for Adaptive Langevin dynamicscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 8.

12 papers.

  1. 2026Can Microcanonical Langevin Dynamics Leverage Mini-Batch Gradient Noise?cited by 6
  2. 2025Randomised Splitting Methods and Stochastic Gradient Descentcited by 3
  3. 2025A Langevin sampling algorithm inspired by the Adam optimizercited by 3
  4. 2025Random Reshuffling for Stochastic Gradient Langevin Dynamicscited by 2
  5. 2026On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriorscited by 2
  6. 2026Towards E-Value Based Stopping Rules for Bayesian Deep Ensemblescited by 2
  7. 2026Position: The Time for Sampling Is Now! Charting a New Course for Bayesian Deep Learningcited by 1
  8. 2026Is the Last Layer Sufficient for Uncertainty Quantification?cited by 1
  9. 2025Scalable Bayesian Monte Carlo: fast uncertainty estimation beyond deep ensemblescited by 0
  10. 2026Kinetic Langevin Splitting Schemes for Constrained Samplingcited by 0
  11. 2026Accurate Large-sample Uncertainty Quantification using Stochastic Gradient Markov Chain Monte Carlocited by 0
  12. 2026Scalable Logistic Gaussian Process Density Regression with Kinetic Langevin Samplingcited by 0