# Martingale Posterior Neural Networks for Fast Sequential Decision Making

arXiv: https://arxiv.org/abs/2506.11898 · HTML: https://mlsanity.com/paper/2506.11898/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 65 references that are in the map.
- First posted on arXiv 13 Jun 2025.

## Builds on (most cited first, 30 of 44)

- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Shampoo: Preconditioned Stochastic Tensor Optimization](https://mlsanity.com/paper/1802.09568/) (2018)
- [Measuring the Intrinsic Dimension of Objective Landscapes](https://mlsanity.com/paper/1804.08838/) (2018)
- [Laplace Redux -- Effortless Bayesian Deep Learning](https://mlsanity.com/paper/2106.14806/) (2021)
- [Efficient and Modular Implicit Differentiation](https://mlsanity.com/paper/2105.15183/) (2021)
- [Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling](https://mlsanity.com/paper/1802.09127/) (2018)
- [Behaviour Suite for Reinforcement Learning](https://mlsanity.com/paper/1908.03568/) (2019)
- [The Bayesian Learning Rule](https://mlsanity.com/paper/2107.04562/) (2021)
- [KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems](https://mlsanity.com/paper/2202.10842/) (2022)
- [Vanilla Bayesian Optimization Performs Great in High Dimensions](https://mlsanity.com/paper/2402.02229/) (2024)
- [On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty](https://mlsanity.com/paper/2102.11409/) (2021)
- ['In-Between' Uncertainty in Bayesian Neural Networks](https://mlsanity.com/paper/1906.11537/) (2019)
- [Variational Bayesian Last Layers](https://mlsanity.com/paper/2404.11599/) (2024)
- [The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents](https://mlsanity.com/paper/2402.03220/) (2024)
- [Neural network learns low-dimensional polynomials with SGD near the information-theoretic limit](https://mlsanity.com/paper/2406.01581/) (2024)
- [Martingale posterior distributions](https://mlsanity.com/paper/2103.15671/) (2021)
- [Repetita Iuvant: Data Repetition Allows SGD to Learn High-Dimensional Multi-Index Functions](https://mlsanity.com/paper/2405.15459/) (2024)
- [SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient](https://mlsanity.com/paper/1811.04504/) (2018)
- [Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates](https://mlsanity.com/paper/2010.00373/) (2020)
- [Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2402.02746/) (2024)
- [Estimating Model Uncertainty of Neural Networks in Sparse Information Form](https://mlsanity.com/paper/2006.11631/) (2020)
- [Monolith: Real Time Recommendation System With Collisionless Embedding Table](https://mlsanity.com/paper/2209.07663/) (2022)
- [How many degrees of freedom do we need to train deep networks: a loss landscape perspective](https://mlsanity.com/paper/2107.05802/) (2021)
- [Online Natural Gradient as a Kalman Filter](https://mlsanity.com/paper/1703.00209/) (2017)
- [Streaming Deep Reinforcement Learning Finally Works](https://mlsanity.com/paper/2410.14606/) (2024)
- [Low-rank extended Kalman filtering for online learning of neural networks from streaming data](https://mlsanity.com/paper/2305.19535/) (2023)
- [Outlier-robust Kalman Filtering through Generalised Bayes](https://mlsanity.com/paper/2405.05646/) (2024)
- [Impatient Bandits: Optimizing Recommendations for the Long-Term Without Delay](https://mlsanity.com/paper/2307.09943/) (2023)
- [Thompson Sampling with Approximate Inference](https://mlsanity.com/paper/1908.04970/) (2019)

## Built on by (most cited first, 4 of 4)

- [A Predictive View on Streaming Hidden Markov Models](https://mlsanity.com/paper/2604.09208/) (2026)
- [Conformal bandits: bringing statistical validity and reward efficiency under weak arm separability](https://mlsanity.com/paper/2512.09850/) (2025)
- [A probabilistic framework for online test-time adaptation](https://mlsanity.com/paper/2606.26457/) (2026)
- [Online Learning via Learned Latent Bayesian Tracking](https://mlsanity.com/paper/2609.31559/) (2026)

## Abstract (arXiv)

We introduce scalable algorithms for online learning of neural network parameters and Bayesian sequential decision making. Unlike classical Bayesian neural networks, which induce predictive uncertainty through a posterior over model parameters, our methods adopt a predictive-first perspective based on martingale posteriors. In particular, we work directly with the one-step-ahead posterior predictive, which we parameterize with a neural network and update sequentially with incoming observations. This decouples Bayesian decision-making from parameter-space inference: we sample from the posterior predictive for decision making, and update the parameters of the posterior predictive via fast, frequentist Kalman-filter-like recursions. Our algorithms operate in a fully online, replay-free setting, providing principled uncertainty quantification without costly posterior sampling. Empirically, they achieve competitive performance-speed trade-offs in non-stationary contextual bandits and Bayesian optimization, offering 10-100 times faster inference than classical Thompson sampling while maintaining comparable or superior decision performance.