# On Feynman--Kac training of partial Bayesian neural networks

arXiv: https://arxiv.org/abs/2310.19608 · HTML: https://mlsanity.com/paper/2310.19608/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 68 references that are in the map.
- First posted on arXiv 30 Oct 2023.

## Builds on (most cited first, 26 of 26)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [What Are Bayesian Neural Network Posteriors Really Like?](https://mlsanity.com/paper/2104.14421/) (2021)
- [Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks](https://mlsanity.com/paper/2002.10118/) (2020)
- [Variational Sequential Monte Carlo](https://mlsanity.com/paper/1705.11140/) (2017)
- [Subspace Inference for Bayesian Deep Learning](https://mlsanity.com/paper/1907.07504/) (2019)
- [Bayesian Deep Learning via Subnetwork Inference](https://mlsanity.com/paper/2010.14689/) (2020)
- [Do Bayesian Neural Networks Need To Be Fully Stochastic?](https://mlsanity.com/paper/2211.06291/) (2022)
- [Differentiable Particle Filtering via Entropy-Regularized Optimal Transport](https://mlsanity.com/paper/2102.07850/) (2021)
- [On Particle Methods for Parameter Estimation in State-Space Models](https://mlsanity.com/paper/1412.8695/) (2014)
- [Challenges in Markov chain Monte Carlo for Bayesian neural networks](https://mlsanity.com/paper/1910.06539/) (2019)
- [Benchmarking the Neural Linear Model for Regression](https://mlsanity.com/paper/1912.08416/) (2019)
- [Automatic Differentiation of Programs with Discrete Randomness](https://mlsanity.com/paper/2210.08572/) (2022)
- [Online Natural Gradient as a Kalman Filter](https://mlsanity.com/paper/1703.00209/) (2017)
- [AMAGOLD: Amortized Metropolis Adjustment for Efficient Stochastic Gradient MCMC](https://mlsanity.com/paper/2003.00193/) (2020)
- [Differentiating Metropolis-Hastings to Optimize Intractable Densities](https://mlsanity.com/paper/2306.07961/) (2023)
- [On nonnegative unbiased estimators](https://mlsanity.com/paper/1309.6473/) (2013)
- [A Tutorial on Parametric Variational Inference](https://mlsanity.com/paper/2301.01236/) (2023)
- [A Global Stochastic Optimization Particle Filter Algorithm](https://mlsanity.com/paper/2007.04803/) (2020)
- [De-biasing particle filtering for a continuous time hidden Markov model with a Cox process observation model](https://mlsanity.com/paper/2206.10478/) (2022)

## Built on by (most cited first, 4 of 4)

- [VIKING: Deep variational inference with stochastic projections](https://mlsanity.com/paper/2510.23684/) (2025)
- [Low Rank Based Subspace Inference for the Laplace Approximation of Bayesian Neural Networks](https://mlsanity.com/paper/2502.02345/) (2025)
- [Diffusion differentiable resampling](https://mlsanity.com/paper/2512.10401/) (2025)
- [Utilising Gradient-Based Proposals Within Sequential Monte Carlo Samplers for Training of Partial Bayesian Neural Networks](https://mlsanity.com/paper/2505.03797/) (2025)

## Abstract (arXiv)

Recently, partial Bayesian neural networks (pBNNs), which only consider a subset of the parameters to be stochastic, were shown to perform competitively with full Bayesian neural networks. However, pBNNs are often multi-modal in the latent variable space and thus challenging to approximate with parametric models. To address this problem, we propose an efficient sampling-based training strategy, wherein the training of a pBNN is formulated as simulating a Feynman--Kac model. We then describe variations of sequential Monte Carlo samplers that allow us to simultaneously estimate the parameters and the latent posterior distribution of this model at a tractable computational cost. Using various synthetic and real-world datasets we show that our proposed training scheme outperforms the state of the art in terms of predictive performance.