mlsanityData to 14 Oct 2026

Efficient Online Bayesian Inference for Neural Bandits

arXiv 2112.00195 ↗First posted 1 Dec 2021cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

In this paper we present a new algorithm for online (sequential) inference in Bayesian neural networks, and show its suitability for tackling contextual bandit problems. The key idea is to combine the extended Kalman filter (which locally linearizes the likelihood function at each time step) with a (learned or random) low-dimensional affine subspace for the parameters; the use of a subspace enables us to scale our algorithm to models with ~ 1M parameters. While most other neural bandit methods need to store the entire past dataset in order to avoid the problem of "catastrophic forgetting", our approach uses constant memory. This is possible because we represent uncertainty about all the parameters in the model, not just the final linear layer. We show good results on the "Deep Bayesian Bandit Showdown" benchmark, as well as MNIST and a recommender system.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 32 of its 58 references that are in the map. First posted on arXiv 1 Dec 2021.

Builds on

32 of 58 references.

References · 32

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  3. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  4. 2015Weight Uncertainty in Neural Networkscited by 554
  5. 2019Introduction to Multi-Armed Banditscited by 481
  6. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  7. 2017A Tutorial on Thompson Samplingcited by 433
  8. 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 370
  9. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 365
  10. 2017Variational Continual Learningcited by 339
  11. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 308
  12. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
20 more
  1. 2018Online Structured Laplace Approximations For Overcoming Catastrophic Forgettingcited by 201
  2. 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 169
  3. 2019Practical Deep Learning with Bayesian Principlescited by 158
  4. 2020When Do Neural Networks Outperform Kernel Methods?cited by 150
  5. 2020Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertaintycited by 129
  6. 2020Neural Thompson Samplingcited by 128
  7. 2019Subspace Inference for Bayesian Deep Learningcited by 95
  8. 2019What Can ResNet Learn Efficiently, Going Beyond Kernels?cited by 91
  9. 2020Bayesian Deep Learning via Subnetwork Inferencecited by 87
  10. 2020The Case for Bayesian Deep Learningcited by 69
  11. 2017The Unreasonable Effectiveness of Structured Random Orthogonal Embeddingscited by 45
  12. 2019On Multi-Armed Bandit Designs for Dose-Finding Clinical Trialscited by 32
  13. 2021How many degrees of freedom do we need to train deep networks: a loss landscape perspectivecited by 28
  14. 2017Online Natural Gradient as a Kalman Filtercited by 25
  15. 2017Action Centered Contextual Banditscited by 23
  16. 2020Deep Bayesian Bandits: Exploring in Online Personalized Recommendationscited by 23
  17. 2017Deep Learning: A Bayesian Perspectivecited by 20
  18. 2019Thompson Sampling with Approximate Inferencecited by 15
  19. 2021Online Limited Memory Neural-Linear Bandits with Likelihood Matchingcited by 14
  20. 2020Exploration in Online Advertising Systems with Deep Uncertainty-Aware Learningcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 9; 2025: 2; 2026: 1.

15 papers.

Sort:

By area:

  1. 2023Low-rank extended Kalman filtering for online learning of neural networks from streaming datacited by 23
  2. 2024Outlier-robust Kalman Filtering through Generalised Bayescited by 21
  3. 2024Scalable Bayesian Learning with posteriorscited by 13
  4. 2024Partially Stochastic Infinitely Deep Bayesian Neural Networkscited by 6
  5. 2024A unifying framework for generalised Bayesian online learning in non-stationary environmentscited by 5
  6. 2025Martingale Posterior Neural Networks for Fast Sequential Decision Makingcited by 4
  7. 2023Detecting Toxic Flowcited by 3
  8. 2024Adaptive Behavioral AI: Reinforcement Learning to Enhance Pharmacy Servicescited by 2
  9. 2025KOALA++: Efficient Kalman-Based Optimization with Gradient-Covariance Productscited by 2
  10. 2022Targeted Advertising on Social Networks Using Online Variational Tensor Regressioncited by 1
  11. 2024The Digital Transformation in Health: How AI Can Improve the Performance of Health Systemscited by 1
  12. 2024Bayesian Bandit Algorithms with Approximate Inference in Stochastic Linear Banditscited by 0
3 more
  1. 2024Optimizing HIV Patient Engagement with Reinforcement Learning in Resource-Limited Settingscited by 0
  2. 2024Adaptive User Journeys in Pharma E-Commerce with Reinforcement Learning: Insights from SwipeRxcited by 0
  3. 2026Subspace Inference Enables Efficient Active Reward Learning from Preferencescited by 0