mlsanityData to 7 Oct 2026

Challenges in Markov chain Monte Carlo for Bayesian neural networks

arXiv 1910.06539 ↗First posted 15 Oct 2019stat.ML cs.LG stat.COcited by 43

Not on the map. Why?

Abstract (arXiv)

Markov chain Monte Carlo (MCMC) methods have not been broadly adopted in Bayesian neural networks (BNNs). This paper initially reviews the main challenges in sampling from the parameter posterior of a neural network via MCMC. Such challenges culminate to lack of convergence to the parameter posterior. Nevertheless, this paper shows that a non-converged Markov chain, generated via MCMC sampling from the parameter space of a neural network, can yield via Bayesian marginalization a valuable posterior predictive distribution of the output of the neural network. Classification examples based on multilayer perceptrons showcase highly accurate posterior predictive distributions. The postulate of limited scope for MCMC developments in BNNs is partially valid; an asymptotically exact parameter posterior seems less plausible, yet an accurate posterior predictive distribution is a tenable research avenue.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 6; 2023: 9; 2024: 10; 2025: 10; 2026: 6.

43 papers.

Sort:

By area:

  1. 2024Variational Bayesian Last Layerscited by 67
  2. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
  3. 2024The Empirical Impact of Neural Parameter Symmetries, or Lack Thereofcited by 21
  4. 2023Towards Efficient MCMC Sampling in Bayesian Neural Networks by Exploiting Symmetrycited by 20
  5. 2022A Comprehensive Review of Digital Twin -- Part 2: Roles of Uncertainty Quantification and Optimization, a Battery Digital Twin, and Perspectivescited by 17
  6. 2024How Interpretable Are Interpretable Graph Neural Networks?cited by 13
  7. 2025Accelerated Parallel Tempering via Neural Transportscited by 13
  8. 2024Sampling from Bayesian Neural Network Posteriors with Symmetric Minibatch Splitting Langevin Dynamicscited by 12
  9. 2022Variational Bayes Deep Operator Network: A data-driven Bayesian solver for parametric differential equationscited by 11
  10. 2025Microcanonical Langevin Ensembles: Advancing the Sampling of Bayesian Neural Networkscited by 10
  11. 2023Scalable Stochastic Gradient Riemannian Langevin Dynamics in Non-Diagonal Metricscited by 8
  12. 2022Uncertainty-Aware Mixed-Variable Machine Learning for Materials Designcited by 6
31 more
  1. 2023Efficient Bayesian Physics Informed Neural Networks for Inverse Problems via Ensemble Kalman Inversioncited by 6
  2. 2023Learning to solve Bayesian inverse problems: An amortized variational inference approach using Gaussian and Flow guidescited by 5
  3. 2023Neural Mixed Effects for Nonlinear Personalized Predictionscited by 5
  4. 2024Bayesian neural networks for predicting uncertainty in full-field material responsecited by 5
  5. 2023Randomized prior wavelet neural operator for uncertainty quantificationcited by 4
  6. 2023On Feynman--Kac training of partial Bayesian neural networkscited by 4
  7. 2022Approximate blocked Gibbs sampling for Bayesian neural networkscited by 3
  8. 2023The Deep Arbitrary Polynomial Chaos Neural Network or how Deep Artificial Neural Networks could benefit from Data-Driven Homogeneous Chaos Theorycited by 3
  9. 2026Richer Bayesian Last Layers with Subsampled NTK Featurescited by 3
  10. 2024Functional Stochastic Gradient MCMC for Bayesian Neural Networkscited by 2
  11. 2025Scalable Bayesian Physics-Informed Kolmogorov-Arnold Networkscited by 2
  12. 2026On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriorscited by 2
  13. 2021Gated Linear Model induced U-net for surrogate modeling and uncertainty quantificationcited by 1
  14. 2022An Ensemble-Based Deep Framework for Estimating Thermo-Chemical State Variables from Flamelet Generated Manifoldscited by 1
  15. 2022Introduction and Exemplars of Uncertainty Decompositioncited by 1
  16. 2025Bayesian Computation in Deep Learningcited by 1
  17. 2026Position: The Time for Sampling Is Now! Charting a New Course for Bayesian Deep Learningcited by 1
  18. 2026FUSE: Quantifying Uncertainty in Vision-Language Models by Bayesian Fusing Epistemic and Aleatoric Uncertaintycited by 1
  19. 2021Dropout Prediction Uncertainty Estimation Using Neuron Activation Strengthcited by 0
  20. 2023Synaptic Sampling of Neural Networkscited by 0
  21. 2024Robust Deep Reinforcement Learning for Inverter-based Volt-Var Control in Partially Observable Distribution Networkscited by 0
  22. 2024Taylor-Sensus Network: Embracing Noise to Enlighten Uncertainty for Scientific Datacited by 0
  23. 2024Stein Variational Newton Neural Network Ensemblescited by 0
  24. 2025Predictive Modeling and Uncertainty Quantification of Fatigue Life in Metal Alloys using Machine Learningcited by 0
  25. 2025Understanding the Trade-offs in Accuracy and Uncertainty Quantification: Architecture and Inference Choices in Bayesian Neural Networkscited by 0
  26. 2025ConBatch-BAL: Batch Bayesian Active Learning under Budget Constraintscited by 0
  27. 2025The Sensitivity of Variational Bayesian Neural Network Performance to Hyperparameterscited by 0
  28. 2025Partial Trace-Class Bayesian Neural Networkscited by 0
  29. 2025Enhancing Multi-Label Thoracic Disease Diagnosis with Deep Ensemble-Based Uncertainty Quantificationcited by 0
  30. 2026Self-Supervised Laplace Approximation for Bayesian Uncertainty Quantificationcited by 0
  31. 2026Ribbon: Scalable Approximation and Robust Uncertainty Quantificationcited by 0