# In all LikelihoodS: How to Reliably Select Pseudo-Labeled Data for Self-Training in Semi-Supervised Learning

arXiv: https://arxiv.org/abs/2303.01117 · HTML: https://mlsanity.com/paper/2303.01117/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 77 references that are in the map.
- First posted on arXiv 2 Mar 2023.

## Builds on (most cited first, 17 of 17)

- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [Bayesian Model Selection, the Marginal Likelihood, and Generalization](https://mlsanity.com/paper/2202.11678/) (2022)
- [Structured Prediction with Partial Labelling through the Infimum Loss](https://mlsanity.com/paper/2003.00920/) (2020)
- [Bayesian Semi-supervised Learning with Graph Gaussian Processes](https://mlsanity.com/paper/1809.04379/) (2018)
- [An Information-theoretical Approach to Semi-supervised Learning under Covariate-shift](https://mlsanity.com/paper/2202.12123/) (2022)
- [Credal Self-Supervised Learning](https://mlsanity.com/paper/2106.11853/) (2021)
- [Seq-UPS: Sequential Uncertainty-aware Pseudo-label Selection for Semi-Supervised Text Recognition](https://mlsanity.com/paper/2209.00641/) (2022)
- [Accounting for Gaussian Process Imprecision in Bayesian Optimization](https://mlsanity.com/paper/2111.08299/) (2021)
- [Information efficient learning of complexly structured preferences: Elicitation procedures and their application to decision making under uncertainty](https://mlsanity.com/paper/2110.12879/) (2021)
- [Safe-Bayesian Generalized Linear Regression](https://mlsanity.com/paper/1910.09227/) (2019)
- [Conformal Credal Self-Supervised Learning](https://mlsanity.com/paper/2205.15239/) (2022)
- [Multi-Target Decision Making under Conditions of Severe Uncertainty](https://mlsanity.com/paper/2212.06832/) (2022)
- [Deep Variational Clustering Framework for Self-labeling of Large-scale Medical Images](https://mlsanity.com/paper/2109.10777/) (2021)
- [Calculating Uncertainty Intervals From Conditional Convex Sets of Probabilities](https://mlsanity.com/paper/1303.5418/) (2013)

## Abstract (arXiv)

Self-training is a simple yet effective method within semi-supervised learning. The idea is to iteratively enhance training data by adding pseudo-labeled data. Its generalization performance heavily depends on the selection of these pseudo-labeled data (PLS). In this paper, we aim at rendering PLS more robust towards the involved modeling assumptions. To this end, we propose to select pseudo-labeled data that maximize a multi-objective utility function. The latter is constructed to account for different sources of uncertainty, three of which we discuss in more detail: model selection, accumulation of errors and covariate shift. In the absence of second-order information on such uncertainties, we furthermore consider the generic approach of the generalized Bayesian alpha-cut updating rule for credal sets. As a practical proof of concept, we spotlight the application of three of our robust extensions on simulated and real-world data. Results suggest that in particular robustness w.r.t. model choice can lead to substantial accuracy gains.