# Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning

arXiv: https://arxiv.org/abs/1907.00865 · HTML: https://mlsanity.com/paper/1907.00865/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jul 2019.

## Built on by (most cited first, 30 of 37)

- [Plex: Towards Reliability using Pretrained Large Model Extensions](https://mlsanity.com/paper/2207.07411/) (2022)
- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [Active Testing: Sample-Efficient Model Evaluation](https://mlsanity.com/paper/2103.05331/) (2021)
- [Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning](https://mlsanity.com/paper/2106.04015/) (2021)
- [Variational Bayesian Last Layers](https://mlsanity.com/paper/2404.11599/) (2024)
- [Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks](https://mlsanity.com/paper/2211.12717/) (2022)
- [Posterior Meta-Replay for Continual Learning](https://mlsanity.com/paper/2103.01133/) (2021)
- [Tractable Function-Space Variational Inference in Bayesian Neural Networks](https://mlsanity.com/paper/2312.17199/) (2023)
- [Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation](https://mlsanity.com/paper/2202.06881/) (2022)
- [Feature Generation and Hypothesis Verification for Reliable Face Anti-Spoofing](https://mlsanity.com/paper/2112.14894/) (2021)
- [On Pathologies in KL-Regularized Reinforcement Learning from Expert Demonstrations](https://mlsanity.com/paper/2212.13936/) (2022)
- [Heavy-tailed denoising score matching](https://mlsanity.com/paper/2112.09788/) (2021)
- [InfoNCE Induces Gaussian Distribution](https://mlsanity.com/paper/2602.24012/) (2026)
- [Riemannian Laplace approximations for Bayesian neural networks](https://mlsanity.com/paper/2306.07158/) (2023)
- [Predictive Churn with the Set of Good Models](https://mlsanity.com/paper/2402.07745/) (2024)
- [Drug Discovery under Covariate Shift with Domain-Informed Prior Distributions over Functions](https://mlsanity.com/paper/2307.15073/) (2023)
- [A Benchmark on Uncertainty Quantification for Deep Learning Prognostics](https://mlsanity.com/paper/2302.04730/) (2023)
- [Uncertainty-aware deep learning methods for robust diabetic retinopathy classification](https://mlsanity.com/paper/2201.09042/) (2022)
- [Bayesian Neural Networks: Essentials](https://mlsanity.com/paper/2106.13594/) (2021)
- [On Sequential Bayesian Inference for Continual Learning](https://mlsanity.com/paper/2301.01828/) (2023)
- [Dual Accuracy-Quality-Driven Neural Network for Prediction Interval Generation](https://mlsanity.com/paper/2212.06370/) (2022)
- [Bayesian Deep Neural Networks for Supervised Learning of Single-View Depth](https://mlsanity.com/paper/2104.14202/) (2021)
- [Graph Reparameterizations for Enabling 1000+ Monte Carlo Iterations in Bayesian Deep Neural Networks](https://mlsanity.com/paper/2202.09478/) (2022)
- [Deep Combinatorial Aggregation](https://mlsanity.com/paper/2210.06436/) (2022)
- [Inadequacy of common stochastic neural networks for reliable clinical decision support](https://mlsanity.com/paper/2401.13657/) (2024)
- [Model and Feature Diversity for Bayesian Neural Networks in Mutual Learning](https://mlsanity.com/paper/2407.02721/) (2024)
- [Trajectory Stitching for Solving Inverse Problems with Flow-Based Models](https://mlsanity.com/paper/2602.08538/) (2026)
- [Efficient Action Recognition Using Confidence Distillation](https://mlsanity.com/paper/2109.02137/) (2021)
- [Spatial-Temporal-Fusion BNN: Variational Bayesian Feature Layer](https://mlsanity.com/paper/2112.06281/) (2021)
- [Uncertainty estimation in Deep Learning for Panoptic segmentation](https://mlsanity.com/paper/2304.02098/) (2023)

## Abstract (arXiv)

We propose Radial Bayesian Neural Networks (BNNs): a variational approximate posterior for BNNs which scales well to large models while maintaining a distribution over weight-space with full support. Other scalable Bayesian deep learning methods, like MC dropout or deep ensembles, have discrete support-they assign zero probability to almost all of the weight-space. Unlike these discrete support methods, Radial BNNs' full support makes them suitable for use as a prior for sequential inference. In addition, they solve the conceptual challenges with the a priori implausibility of weight distributions with discrete support. The Radial BNN is motivated by avoiding a sampling problem in 'mean-field' variational inference (MFVI) caused by the so-called 'soap-bubble' pathology of multivariate Gaussians. We show that, unlike MFVI, Radial BNNs are robust to hyperparameters and can be efficiently applied to a challenging real-world medical application without needing ad-hoc tweaks and intensive tuning. In fact, in this setting Radial BNNs out-perform discrete-support methods like MC dropout. Lastly, by using Radial BNNs as a theoretically principled, robust alternative to MFVI we make significant strides in a Bayesian continual learning evaluation.