# Automated Scalable Bayesian Inference via Hilbert Coresets

arXiv: https://arxiv.org/abs/1710.05053 · HTML: https://mlsanity.com/paper/1710.05053/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2017.

## Built on by (most cited first, 30 of 55)

- [Online Coreset Selection for Rehearsal-based Continual Learning](https://mlsanity.com/paper/2106.01085/) (2021)
- [GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning](https://mlsanity.com/paper/2111.11210/) (2021)
- [Graph Condensation via Receptive Field Distribution Matching](https://mlsanity.com/paper/2206.13697/) (2022)
- [Kernel Thinning](https://mlsanity.com/paper/2105.05842/) (2021)
- [Information-theoretic Online Memory Selection for Continual Learning](https://mlsanity.com/paper/2204.04763/) (2022)
- [The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization](https://mlsanity.com/paper/2206.02768/) (2022)
- [Can Public Large Language Models Help Private Cross-device Federated Learning?](https://mlsanity.com/paper/2305.12132/) (2023)
- [Fast Bayesian Inference with Batch Bayesian Quadrature via Kernel Recombination](https://mlsanity.com/paper/2206.04734/) (2022)
- [Surrogate Likelihoods for Variational Annealed Importance Sampling](https://mlsanity.com/paper/2112.12194/) (2021)
- [GRASP: A Rehearsal Policy for Efficient Online Continual Learning](https://mlsanity.com/paper/2308.13646/) (2023)
- [Bayesian inference via sparse Hamiltonian flows](https://mlsanity.com/paper/2203.05723/) (2022)
- [Data pruning and neural scaling laws: fundamental limitations of score-based algorithms](https://mlsanity.com/paper/2302.06960/) (2023)
- [Selectivity Drives Productivity: Efficient Dataset Pruning for Enhanced Transfer Learning](https://mlsanity.com/paper/2310.08782/) (2023)
- [On Divergence Measures for Bayesian Pseudocoresets](https://mlsanity.com/paper/2210.06205/) (2022)
- [Bayesian Active Learning by Disagreements: A Geometric Perspective](https://mlsanity.com/paper/2105.02543/) (2021)
- [Fast Bayesian Coresets via Subsampling and Quasi-Newton Refinement](https://mlsanity.com/paper/2203.09675/) (2022)
- [Beyond Efficiency: Molecular Data Pruning for Enhanced Generalization](https://mlsanity.com/paper/2409.01081/) (2024)
- [Data Summarization via Bilevel Optimization](https://mlsanity.com/paper/2109.12534/) (2021)
- [Two Trades is not Baffled: Condensing Graph via Crafting Rational Gradient Matching](https://mlsanity.com/paper/2402.04924/) (2024)
- [Posterior Coreset Construction with Kernelized Stein Discrepancy for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2206.01162/) (2022)
- [Adaptive Batch Sizes for Active Learning A Probabilistic Numerics Approach](https://mlsanity.com/paper/2306.05843/) (2023)
- [Coreset Markov Chain Monte Carlo](https://mlsanity.com/paper/2310.17063/) (2023)
- [One Backward from Ten Forward, Subsampling for Large-Scale Deep Learning](https://mlsanity.com/paper/2104.13114/) (2021)
- [Black-box Coreset Variational Inference](https://mlsanity.com/paper/2211.02377/) (2022)
- [Robustness-preserving Lifelong Learning via Dataset Condensation](https://mlsanity.com/paper/2303.04183/) (2023)
- [Bayesian Pseudo-Coresets via Contrastive Divergence](https://mlsanity.com/paper/2303.11278/) (2023)
- [Machine Learning and the Future of Bayesian Computation](https://mlsanity.com/paper/2304.11251/) (2023)
- [Structural-Entropy-Based Sample Selection for Efficient and Effective Learning](https://mlsanity.com/paper/2410.02268/) (2024)
- [Aligning Compound AI Systems via System-level DPO](https://mlsanity.com/paper/2502.17721/) (2025)
- [Marginal and Joint Cross-Entropies & Predictives for Online Bayesian Inference, Active Learning, and Active Sampling](https://mlsanity.com/paper/2205.08766/) (2022)

## Abstract (arXiv)

The automation of posterior inference in Bayesian data analysis has enabled experts and nonexperts alike to use more sophisticated models, engage in faster exploratory modeling and analysis, and ensure experimental reproducibility. However, standard automated posterior inference algorithms are not tractable at the scale of massive modern datasets, and modifications to make them so are typically model-specific, require expert tuning, and can break theoretical guarantees on inferential quality. Building on the Bayesian coresets framework, this work instead takes advantage of data redundancy to shrink the dataset itself as a preprocessing step, providing fully-automated, scalable Bayesian inference with theoretical guarantees. We begin with an intuitive reformulation of Bayesian coreset construction as sparse vector sum approximation, and demonstrate that its automation and performance-based shortcomings arise from the use of the supremum norm. To address these shortcomings we develop Hilbert coresets, i.e., Bayesian coresets constructed under a norm induced by an inner-product on the log-likelihood function space. We propose two Hilbert coreset construction algorithms---one based on importance sampling, and one based on the Frank-Wolfe algorithm---along with theoretical guarantees on approximation quality as a function of coreset size. Since the exact computation of the proposed inner-products is model-specific, we automate the construction with a random finite-dimensional projection of the log-likelihood functions. The resulting automated coreset construction algorithm is simple to implement, and experiments on a variety of models with real and synthetic datasets show that it provides high-quality posterior approximations and a significant reduction in the computational cost of inference.