# Investigating Batch Inference in a Sequential Monte Carlo Framework for Neural Networks

arXiv: https://arxiv.org/abs/2601.21983 · HTML: https://mlsanity.com/paper/2601.21983/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 20 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Builds on (most cited first, 6 of 6)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [What Are Bayesian Neural Network Posteriors Really Like?](https://mlsanity.com/paper/2104.14421/) (2021)
- [Enhanced SMC2: Leveraging Gradient Information from Differentiable Particle Filters Within Langevin Proposals](https://mlsanity.com/paper/2407.17296/) (2024)
- [Incorporating the ChEES Criterion into Sequential Monte Carlo Samplers](https://mlsanity.com/paper/2504.02627/) (2025)
- [Hess-MC2: Sequential Monte Carlo Squared using Hessian Information and Second Order Proposals](https://mlsanity.com/paper/2507.07461/) (2025)

## Abstract (arXiv)

Bayesian inference allows us to define a posterior distribution over the weights of a generic neural network (NN). Exact posteriors are usually intractable, in which case approximations can be employed. One such approximation - variational inference - is computationally efficient when using mini-batch stochastic gradient descent as subsets of the data are used for likelihood and gradient evaluations, though the approach relies on the selection of a variational distribution which sufficiently matches the form of the posterior. Particle-based methods such as Markov chain Monte Carlo and Sequential Monte Carlo (SMC) do not assume a parametric family for the posterior by typically require higher computational cost. These sampling methods typically use the full-batch of data for likelihood and gradient evaluations, which contributes to this computational expense. We explore several methods of gradually introducing more mini-batches of data (data annealing) into likelihood and gradient evaluations of an SMC sampler. We find that we can achieve up to 6× faster training with minimal loss in accuracy on benchmark image classification problems using NNs.