# Tractable Function-Space Variational Inference in Bayesian Neural Networks

arXiv: https://arxiv.org/abs/2312.17199 · HTML: https://mlsanity.com/paper/2312.17199/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 62 references that are in the map.
- First posted on arXiv 28 Dec 2023.

## Builds on (most cited first, 26 of 26)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Unsolved Problems in ML Safety](https://mlsanity.com/paper/2109.13916/) (2021)
- [Practical Deep Learning with Bayesian Principles](https://mlsanity.com/paper/1906.02506/) (2019)
- [Functional Variational Bayesian Neural Networks](https://mlsanity.com/paper/1903.05779/) (2019)
- [Functional Regularisation for Continual Learning with Gaussian Processes](https://mlsanity.com/paper/1901.11356/) (2019)
- [Continual Deep Learning by Functional Regularisation of Memorable Past](https://mlsanity.com/paper/2004.14070/) (2020)
- [Measuring and regularizing networks in function space](https://mlsanity.com/paper/1805.08289/) (2018)
- [Subspace Inference for Bayesian Deep Learning](https://mlsanity.com/paper/1907.07504/) (2019)
- [Approximate Inference Turns Deep Networks into Gaussian Processes](https://mlsanity.com/paper/1906.01930/) (2019)
- ['In-Between' Uncertainty in Bayesian Neural Networks](https://mlsanity.com/paper/1906.11537/) (2019)
- [A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks](https://mlsanity.com/paper/1912.10481/) (2019)
- [On Sparse variational methods and the Kullback-Leibler divergence between stochastic processes](https://mlsanity.com/paper/1504.07027/) (2015)
- [Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks](https://mlsanity.com/paper/2211.12717/) (2022)
- [Global inducing point variational posteriors for Bayesian neural networks and deep Gaussian processes](https://mlsanity.com/paper/2005.08140/) (2020)
- [Variational Implicit Processes](https://mlsanity.com/paper/1806.02390/) (2018)
- [Function Space Particle Optimization for Bayesian Neural Networks](https://mlsanity.com/paper/1902.09754/) (2019)
- [Understanding Variational Inference in Function-Space](https://mlsanity.com/paper/2011.09421/) (2020)
- [Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning](https://mlsanity.com/paper/1907.00865/) (2019)
- [On Pathologies in KL-Regularized Reinforcement Learning from Expert Demonstrations](https://mlsanity.com/paper/2212.13936/) (2022)
- [Scalable Uncertainty for Computer Vision with Functional Variational Inference](https://mlsanity.com/paper/2003.03396/) (2020)
- [Improving predictions of Bayesian neural nets via local linearization](https://mlsanity.com/paper/2008.08400/) (2020)

## Built on by (most cited first, 30 of 41)

- [Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI](https://mlsanity.com/paper/2402.00809/) (2024)
- [CreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Tasks](https://mlsanity.com/paper/2401.05043/) (2024)
- [Position: The Future of Bayesian Prediction Is Prior-Fitted](https://mlsanity.com/paper/2505.23947/) (2025)
- [Uncertainty Quantification for Machine Learning in Healthcare: A Survey](https://mlsanity.com/paper/2505.02874/) (2025)
- [Out-of-Distribution Detection Methods Answer the Wrong Questions](https://mlsanity.com/paper/2507.01831/) (2025)
- [Credal Wrapper of Model Averaging for Uncertainty Estimation in Classification](https://mlsanity.com/paper/2405.15047/) (2024)
- [Context-Guided Diffusion for Out-of-Distribution Molecular and Protein Design](https://mlsanity.com/paper/2407.11942/) (2024)
- [FSP-Laplace: Function-Space Priors for the Laplace Approximation in Bayesian Deep Learning](https://mlsanity.com/paper/2407.13711/) (2024)
- [Posterior Uncertainty Quantification in Neural Networks using Data Augmentation](https://mlsanity.com/paper/2403.12729/) (2024)
- [Function Space Diversity for Uncertainty Prediction via Repulsive Last-Layer Ensembles](https://mlsanity.com/paper/2412.15758/) (2024)
- [Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'](https://mlsanity.com/paper/2505.04950/) (2025)
- [Variational Inference Failures Under Model Symmetries: Permutation Invariant Posteriors for Bayesian Neural Networks](https://mlsanity.com/paper/2408.05496/) (2024)
- [BayTTA: Uncertainty-aware medical image classification with optimized test-time augmentation using Bayesian model averaging](https://mlsanity.com/paper/2406.17640/) (2024)
- [Low-Budget Simulation-Based Inference with Bayesian Neural Networks](https://mlsanity.com/paper/2408.15136/) (2024)
- [Deep Exploration with PAC-Bayes](https://mlsanity.com/paper/2402.03055/) (2024)
- [Spectral Convolutional Conditional Neural Processes](https://mlsanity.com/paper/2404.13182/) (2024)
- [Contextual Similarity Distillation: Ensemble Uncertainties with a Single Model](https://mlsanity.com/paper/2503.11339/) (2025)
- [Uncertainty in Physics and AI: Taxonomy, Quantification, and Validation](https://mlsanity.com/paper/2605.10378/) (2026)
- [Attacking Bayes: On the Adversarial Robustness of Bayesian Neural Networks](https://mlsanity.com/paper/2404.19640/) (2024)
- [Generative vs. Discriminative modeling under the lens of uncertainty quantification](https://mlsanity.com/paper/2406.09172/) (2024)
- [Functional Stochastic Gradient MCMC for Bayesian Neural Networks](https://mlsanity.com/paper/2409.16632/) (2024)
- [R'enyi Neural Processes](https://mlsanity.com/paper/2405.15991/) (2024)
- [One-Shot Federated Learning with Bayesian Pseudocoresets](https://mlsanity.com/paper/2406.02177/) (2024)
- [Maxitive Donsker-Varadhan Formulation for Possibilistic Variational Inference](https://mlsanity.com/paper/2511.21223/) (2025)
- [Bayesian PINNs for uncertainty-aware inverse problems (BPINN-IP)](https://mlsanity.com/paper/2602.04459/) (2026)
- [A neurosymbolic Approach with Epistemic Deep Learning for Hierarchical Image Classification](https://mlsanity.com/paper/2605.16383/) (2026)
- [Flow-Transformed Implicit Processes for Function-Space Variational Inference](https://mlsanity.com/paper/2606.01954/) (2026)
- [On Uniform, Bayesian, and PAC-Bayesian Deep Ensembles](https://mlsanity.com/paper/2406.05469/) (2024)
- [Feature Preserving Shrinkage on Bayesian Neural Networks via the R2D2 Prior](https://mlsanity.com/paper/2505.18280/) (2025)
- [Variational Deep Learning via Implicit Regularization](https://mlsanity.com/paper/2505.20235/) (2025)

## Abstract (arXiv)

Reliable predictive uncertainty estimation plays an important role in enabling the deployment of neural networks to safety-critical settings. A popular approach for estimating the predictive uncertainty of neural networks is to define a prior distribution over the network parameters, infer an approximate posterior distribution, and use it to make stochastic predictions. However, explicit inference over neural network parameters makes it difficult to incorporate meaningful prior information about the data-generating process into the model. In this paper, we pursue an alternative approach. Recognizing that the primary object of interest in most settings is the distribution over functions induced by the posterior distribution over neural network parameters, we frame Bayesian inference in neural networks explicitly as inferring a posterior distribution over functions and propose a scalable function-space variational inference method that allows incorporating prior information and results in reliable predictive uncertainty estimates. We show that the proposed method leads to state-of-the-art uncertainty estimation and predictive performance on a range of prediction tasks and demonstrate that it performs well on a challenging safety-critical medical diagnosis task in which reliable uncertainty estimation is essential.