mlsanityData to 14 Oct 2026

Utilising Gradient-Based Proposals Within Sequential Monte Carlo Samplers for Training of Partial Bayesian Neural Networks

arXiv 2505.03797 ↗First posted 1 May 2025cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Partial Bayesian neural networks (pBNNs) have been shown to perform competitively with fully Bayesian neural networks while only having a subset of the parameters be stochastic. Using sequential Monte Carlo (SMC) samplers as the inference method for pBNNs gives a non-parametric probabilistic estimation of the stochastic parameters, and has shown improved performance over parametric methods. In this paper we introduce a new SMC-based training method for pBNNs by utilising a guided proposal and incorporating gradient-based Markov kernels, which gives us better scalability on high dimensional problems. We show that our new method outperforms the state-of-the-art in terms of predictive performance and optimal loss. We also show that pBNNs scale well with larger batch sizes, resulting in significantly reduced training times and often better performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 44 references that are in the map. First posted on arXiv 1 May 2025.

Builds on

17 of 44 references.

References · 17

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  5. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  6. 2017Deep Bayesian Active Learning with Image Datacited by 819
  7. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
  8. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 343
  9. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 308
  10. 2017Stein Variational Gradient Descent as Gradient Flowcited by 152
  11. 2022Do Bayesian Neural Networks Need To Be Fully Stochastic?cited by 63
  12. 2019Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networkscited by 45
5 more
  1. 2020An invitation to sequential Monte Carlo samplerscited by 33
  2. 2024The Empirical Impact of Neural Parameter Symmetries, or Lack Thereofcited by 21
  3. 2023On Feynman--Kac training of partial Bayesian neural networkscited by 4
  4. 2024Enhanced SMC2: Leveraging Gradient Information from Differentiable Particle Filters Within Langevin Proposalscited by 4
  5. 2025Incorporating the ChEES Criterion into Sequential Monte Carlo Samplerscited by 3

Built on by

Not cited yet.