mlsanityData to 7 Oct 2026

Functional Variational Bayesian Neural Networks

arXiv 1903.05779 ↗First posted 14 Mar 2019cs.LG stat.MLcited by 127

Not on the map. Why?

Abstract (arXiv)

Variational Bayesian neural networks (BNNs) perform variational inference over weights, but it is difficult to specify meaningful priors and approximate posteriors in a high-dimensional weight space. We introduce functional variational Bayesian neural networks (fBNNs), which maximize an Evidence Lower BOund (ELBO) defined directly on stochastic processes, i.e. distributions over functions. We prove that the KL divergence between stochastic processes equals the supremum of marginal KL divergences over all finite sets of inputs. Based on this, we introduce a practical training objective which approximates the functional ELBO using finite measurement sets and the spectral Stein gradient estimator. With fBNNs, we can specify priors entailing rich structures, including Gaussian processes and implicit stochastic processes. Empirically, we find fBNNs extrapolate well using various structured priors, provide reliable uncertainty estimates, and scale to large datasets.

Key facts. Cited by 127 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 32; 2022: 19; 2023: 28; 2024: 17; 2025: 15; 2026: 15.

127 papers.

Sort:

By area:

  1. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  2. 2021Bayesian Neural Network Priors Revisitedcited by 121
  3. 2021Epistemic Neural Networkscited by 120
  4. 2021Repulsive Deep Ensembles are Bayesiancited by 103
  5. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  6. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferencescited by 73
  7. 2024Variational Bayesian Last Layerscited by 67
  8. 2021Pre-trained Gaussian Processes for Bayesian Optimizationcited by 63
  9. 2022Do Bayesian Neural Networks Need To Be Fully Stochastic?cited by 63
  10. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
  11. 2022Diffusion Generative Models in Infinite Dimensionscited by 49
  12. 2023Tractable Function-Space Variational Inference in Bayesian Neural Networkscited by 41
38 more
  1. 2021LiBRe: A Practical Bayesian Approach to Adversarial Detectioncited by 40
  2. 2021Continuous-Time Model-Based Reinforcement Learningcited by 39
  3. 2021The Gaussian Neural Processcited by 34
  4. 2023BayesDAG: Gradient-Based Posterior Inference for Causal Discoverycited by 33
  5. 2023A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methodscited by 31
  6. 2023Continual Learning via Sequential Function-Space Variational Inferencecited by 28
  7. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 26
  8. 2021Simultaneously Reconciled Quantile Forecasting of Hierarchically Related Time Seriescited by 24
  9. 2022Spectral Diffusion Processescited by 24
  10. 2021Meta-Learning Reliable Priors in the Function Spacecited by 23
  11. 2021The Neural Testbed: Evaluating Joint Predictionscited by 22
  12. 2022Deep Learning and Symbolic Regression for Discovering Parametric Equationscited by 22
  13. 2023Bayesian Federated Learning: A Surveycited by 22
  14. 2021On Stein Variational Neural Network Ensemblescited by 21
  15. 2021Periodic Activation Functions Induce Stationaritycited by 21
  16. 2023Coherent Soft Imitation Learningcited by 21
  17. 2022Generalized Variational Inference in Function Spaces: Gaussian Measures meet Bayesian Deep Learningcited by 20
  18. 2023Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shiftcited by 20
  19. 2023Sampling weights of deep neural networkscited by 20
  20. 2023Function-Space Regularization in Neural Networks: A Probabilistic Perspectivecited by 20
  21. 2023Random-Set Neural Networks (RS-NN)cited by 19
  22. 2021Sparse Uncertainty Representation in Deep Learning with Inducing Weightscited by 16
  23. 2022Amortized Variational Inference: A Systematic Reviewcited by 16
  24. 2024Sparse Inducing Points in Deep Gaussian Processes: Enhancing Modeling with Denoising Diffusion Variational Inferencecited by 16
  25. 2021Deep Learning for Bayesian Optimization of Scientific Problems with High-Dimensional Structurecited by 15
  26. 2022Posterior Refinement Improves Sample Efficiency in Bayesian Neural Networkscited by 14
  27. 2021Function-space Inference with Sparse Implicit Processescited by 13
  28. 2023A Symmetry-Aware Exploration of Bayesian Neural Network Posteriorscited by 12
  29. 2024FSP-Laplace: Function-Space Priors for the Laplace Approximation in Bayesian Deep Learningcited by 11
  30. 2021Model Selection for Bayesian Autoencoderscited by 10
  31. 2023A Scalable Walsh-Hadamard Regularizer to Overcome the Low-degree Spectral Bias of Neural Networkscited by 10
  32. 2023Drug Discovery under Covariate Shift with Domain-Informed Prior Distributions over Functionscited by 10
  33. 2023Incorporating Unlabelled Data into Bayesian Neural Networkscited by 9
  34. 2021Training on Test Data with Bayesian Adaptation for Covariate Shiftcited by 8
  35. 2022Tackling covariate shift with node-based Bayesian neural networkscited by 8
  36. 2022Deep Variational Implicit Processescited by 8
  37. 2022On the detrimental effect of invariances in the likelihood for variational inferencecited by 8
  38. 2023Random Grid Neural Processes for Parametric Partial Differential Equationscited by 8

50 most cited of 127 All 127 (JSON)