# Sparse Variational Inference: Bayesian Coresets from Scratch

arXiv: https://arxiv.org/abs/1906.03329 · HTML: https://mlsanity.com/paper/1906.03329/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2019.

## Built on by (most cited first, 28 of 28)

- [GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning](https://mlsanity.com/paper/2111.11210/) (2021)
- [Fast Bayesian Inference with Batch Bayesian Quadrature via Kernel Recombination](https://mlsanity.com/paper/2206.04734/) (2022)
- [Bayesian inference via sparse Hamiltonian flows](https://mlsanity.com/paper/2203.05723/) (2022)
- [On Divergence Measures for Bayesian Pseudocoresets](https://mlsanity.com/paper/2210.06205/) (2022)
- [Fast Bayesian Coresets via Subsampling and Quasi-Newton Refinement](https://mlsanity.com/paper/2203.09675/) (2022)
- [A Benchmark and Empirical Analysis for Replay Strategies in Continual Learning](https://mlsanity.com/paper/2208.02660/) (2022)
- [A Novel Sequential Coreset Method for Gradient Descent Algorithms](https://mlsanity.com/paper/2112.02504/) (2021)
- [Coreset Markov Chain Monte Carlo](https://mlsanity.com/paper/2310.17063/) (2023)
- [Black-box Coreset Variational Inference](https://mlsanity.com/paper/2211.02377/) (2022)
- [Bayesian Pseudo-Coresets via Contrastive Divergence](https://mlsanity.com/paper/2303.11278/) (2023)
- [Machine Learning and the Future of Bayesian Computation](https://mlsanity.com/paper/2304.11251/) (2023)
- [Poisoning Bayesian Inference via Data Deletion and Replication](https://mlsanity.com/paper/2503.04480/) (2025)
- [Function Space Bayesian Pseudocoreset for Bayesian Neural Networks](https://mlsanity.com/paper/2310.17852/) (2023)
- [General bounds on the quality of Bayesian coresets](https://mlsanity.com/paper/2405.11780/) (2024)
- [Reproducible, incremental representation learning with Rosetta VAE](https://mlsanity.com/paper/2201.05206/) (2022)
- [Coreset of Hyperspectral Images on Small Quantum Computer](https://mlsanity.com/paper/2204.04691/) (2022)
- [Bayes Hilbert Spaces for Posterior Approximation](https://mlsanity.com/paper/2304.09053/) (2023)
- [Uncertainty Aware Neural Network from Similarity and Sensitivity](https://mlsanity.com/paper/2304.14925/) (2023)
- [Bayesian Coreset Optimization for Personalized Federated Learning](https://mlsanity.com/paper/2511.01800/) (2025)
- [How To Train Your Program: a Probabilistic Programming Pattern for Bayesian Learning From Data](https://mlsanity.com/paper/2105.03650/) (2021)
- [Centroid Approximation for Bootstrap: Improving Particle Quality at Inference](https://mlsanity.com/paper/2110.08720/) (2021)
- [Training Quantum Boltzmann Machines with Coresets](https://mlsanity.com/paper/2307.14459/) (2023)
- [Efficient Incremental Belief Updates Using Weighted Virtual Observations](https://mlsanity.com/paper/2402.06940/) (2024)
- [MCMC-driven learning](https://mlsanity.com/paper/2402.09598/) (2024)
- [Tuning-Free Coreset Markov Chain Monte Carlo via Hot DoG](https://mlsanity.com/paper/2410.18973/) (2024)
- [Variational Bayesian Pseudo-Coreset](https://mlsanity.com/paper/2502.21143/) (2025)
- [Position: Stop Preaching and Start Practising Data Frugality for Responsible Development of AI](https://mlsanity.com/paper/2602.19789/) (2026)
- [Joint Model and Data Sparsification via the Marginal Likelihood](https://mlsanity.com/paper/2605.29908/) (2026)

## Abstract (arXiv)

The proliferation of automated inference algorithms in Bayesian statistics has provided practitioners newfound access to fast, reproducible data analysis and powerful statistical models. Designing automated methods that are also both computationally scalable and theoretically sound, however, remains a significant challenge. Recent work on Bayesian coresets takes the approach of compressing the dataset before running a standard inference algorithm, providing both scalability and guarantees on posterior approximation error. But the automation of past coreset methods is limited because they depend on the availability of a reasonable coarse posterior approximation, which is difficult to specify in practice. In the present work we remove this requirement by formulating coreset construction as sparsity-constrained variational inference within an exponential family. This perspective leads to a novel construction via greedy optimization, and also provides a unifying information-geometric view of present and past methods. The proposed Riemannian coreset construction algorithm is fully automated, requiring no problem-specific inputs aside from the probabilistic model and dataset. In addition to being significantly easier to use than past methods, experiments demonstrate that past coreset constructions are fundamentally limited by the fixed coarse posterior approximation; in contrast, the proposed algorithm is able to continually improve the coreset, providing state-of-the-art Bayesian dataset summarization with orders-of-magnitude reduction in KL divergence to the exact posterior.