# Towards Bayesian Data Selection

arXiv: https://arxiv.org/abs/2406.12560 · HTML: https://mlsanity.com/paper/2406.12560/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 18 Jun 2024.

## Builds on (most cited first, 9 of 9)

- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Data Selection for Language Models via Importance Resampling](https://mlsanity.com/paper/2302.03169/) (2023)
- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [Statistical Comparisons of Classifiers by Generalized Stochastic Dominance](https://mlsanity.com/paper/2209.01857/) (2022)
- [Accounting for Gaussian Process Imprecision in Bayesian Optimization](https://mlsanity.com/paper/2111.08299/) (2021)
- [Robust Statistical Comparison of Random Variables with Locally Varying Scale of Measurement](https://mlsanity.com/paper/2306.12803/) (2023)
- [Approximately Bayes-Optimal Pseudo Label Selection](https://mlsanity.com/paper/2302.08883/) (2023)
- [Semi-Supervised Learning guided by the Generalized Bayes Rule under Soft Revision](https://mlsanity.com/paper/2405.15294/) (2024)

## Built on by (most cited first, 2 of 2)

- [Generalization Bounds and Stopping Rules for Learning with Self-Selected Data](https://mlsanity.com/paper/2505.07367/) (2025)
- [Self-Supervised Laplace Approximation for Bayesian Uncertainty Quantification](https://mlsanity.com/paper/2605.12208/) (2026)

## Abstract (arXiv)

A wide range of machine learning algorithms iteratively add data to the training sample. Examples include semi-supervised learning, active learning, multi-armed bandits, and Bayesian optimization. We embed this kind of data addition into decision theory by framing data selection as a decision problem. This paves the way for finding Bayes-optimal selections of data. For the illustrative case of self-training in semi-supervised learning, we derive the respective Bayes criterion. We further show that deploying this criterion mitigates the issue of confirmation bias by empirically assessing our method for generalized linear models, semi-parametric generalized additive models, and Bayesian neural networks on simulated and real-world data.