# Robust and Scalable Variational Bayes

arXiv: https://arxiv.org/abs/2504.12528 · HTML: https://mlsanity.com/paper/2504.12528/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 42 references that are in the map.
- First posted on arXiv 16 Apr 2025.

## Builds on (most cited first, 18 of 18)

- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Concentration of tempered posteriors and of their variational approximations](https://mlsanity.com/paper/1706.09293/) (2017)
- [Frequentist Consistency of Variational Bayes](https://mlsanity.com/paper/1705.03439/) (2017)
- [Loss minimization and parameter estimation with heavy tails](https://mlsanity.com/paper/1307.1827/) (2013)
- [Convergence Rates of Variational Posterior Distributions](https://mlsanity.com/paper/1712.02519/) (2017)
- [Scalable Bayes via Barycenter in Wasserstein Space](https://mlsanity.com/paper/1508.05880/) (2015)
- [Variational Bayes for high-dimensional linear regression with sparse priors](https://mlsanity.com/paper/1904.07150/) (2019)
- [Covariances, Robustness, and Variational Bayes](https://mlsanity.com/paper/1709.02536/) (2017)
- [Adaptive variational Bayes: Optimality, computation and applications](https://mlsanity.com/paper/2109.03204/) (2021)
- [Variational Inference based on Robust Divergences](https://mlsanity.com/paper/1710.06595/) (2017)
- [Parallelizing MCMC with Random Partition Trees](https://mlsanity.com/paper/1506.03164/) (2015)
- [Variational Inference for Infinitely Deep Neural Networks](https://mlsanity.com/paper/2209.10091/) (2022)
- [Robust and Scalable Bayes via a Median of Subset Posterior Measures](https://mlsanity.com/paper/1403.2660/) (2014)
- [Bayesian neural networks and dimensionality reduction](https://mlsanity.com/paper/2008.08044/) (2020)
- [Variational Bayes for high-dimensional proportional hazards models with applications within gene expression](https://mlsanity.com/paper/2112.10270/) (2021)
- [Spatial Multivariate Trees for Big Data Bayesian Regression](https://mlsanity.com/paper/2012.00943/) (2020)
- [Natural Gradient Variational Inference with Gaussian Mixture Models](https://mlsanity.com/paper/2111.08002/) (2021)

## Built on by (most cited first, 1 of 1)

- [Robust Variational Bayes by Min-Max Median Aggregation](https://mlsanity.com/paper/2512.12676/) (2025)

## Abstract (arXiv)

We propose a robust and scalable framework for variational Bayes (VB) that effectively handles outliers and contamination of arbitrary nature in large datasets. Our approach divides the dataset into disjoint subsets, computes the posterior for each subset, and applies VB approximation independently to these posteriors. The resulting variational posteriors with respect to the subsets are then aggregated using the geometric median of probability measures, computed with respect to the Wasserstein distance. This novel aggregation method yields the Variational Median Posterior (VM-Posterior) distribution. We rigorously demonstrate that the VM-Posterior preserves contraction properties akin to those of the true posterior, while accounting for approximation errors or the variational gap inherent in VB methods. We also provide provable robustness guarantee of the VM-Posterior. Furthermore, we establish a variational Bernstein-von Mises theorem for both multivariate Gaussian distributions with general covariance structures and the mean-field variational family. To facilitate practical implementation, we adapt existing algorithms for computing the VM-Posterior and evaluate its performance through extensive numerical experiments. The results highlight its robustness and scalability, making it a reliable tool for Bayesian inference in the presence of complex, contaminated datasets.