# Bayesian data selection

arXiv: https://arxiv.org/abs/2109.02712 · HTML: https://mlsanity.com/paper/2109.02712/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2021.

## Built on by (most cited first, 6 of 6)

- [Learning Robust Statistics for Simulation-based Inference under Model Misspecification](https://mlsanity.com/paper/2305.15871/) (2023)
- [Simulation-based Bayesian inference under model misspecification](https://mlsanity.com/paper/2503.12315/) (2025)
- [Biological Sequence Kernels with Guaranteed Flexibility](https://mlsanity.com/paper/2304.03775/) (2023)
- [Semi-Supervised Learning guided by the Generalized Bayes Rule under Soft Revision](https://mlsanity.com/paper/2405.15294/) (2024)
- [Reciprocal Learning](https://mlsanity.com/paper/2408.06257/) (2024)
- [Bilateral Distribution Compression: Reducing Both Data Size and Dimensionality](https://mlsanity.com/paper/2509.17543/) (2025)

## Abstract (arXiv)

Insights into complex, high-dimensional data can be obtained by discovering features of the data that match or do not match a model of interest. To formalize this task, we introduce the "data selection" problem: finding a lower-dimensional statistic - such as a subset of variables - that is well fit by a given parametric model of interest. A fully Bayesian approach to data selection would be to parametrically model the value of the statistic, nonparametrically model the remaining "background" components of the data, and perform standard Bayesian model selection for the choice of statistic. However, fitting a nonparametric model to high-dimensional data tends to be highly inefficient, statistically and computationally. We propose a novel score for performing both data selection and model selection, the "Stein volume criterion", that takes the form of a generalized marginal likelihood with a kernelized Stein discrepancy in place of the Kullback-Leibler divergence. The Stein volume criterion does not require one to fit or even specify a nonparametric background model, making it straightforward to compute - in many cases it is as simple as fitting the parametric model of interest with an alternative objective function. We prove that the Stein volume criterion is consistent for both data selection and model selection, and we establish consistency and asymptotic normality (Bernstein-von Mises) of the corresponding generalized posterior on parameters. We validate our method in simulation and apply it to the analysis of single-cell RNA sequencing datasets using probabilistic principal components analysis and a spin glass model of gene regulation.