# Multi-class Probabilistic Bounds for Self-learning

arXiv: https://arxiv.org/abs/2109.14422 · HTML: https://mlsanity.com/paper/2109.14422/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 57 references that are in the map.
- First posted on arXiv 29 Sep 2021.

## Builds on (most cited first, 12 of 12)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Are Anchor Points Really Indispensable in Label-Noise Learning?](https://mlsanity.com/paper/1906.00189/) (2019)
- [Understanding Random Forests: From Theory to Practice](https://mlsanity.com/paper/1407.7502/) (2014)
- [A Strongly Quasiconvex PAC-Bayesian Bound](https://mlsanity.com/paper/1608.05610/) (2016)
- [Risk Bounds for the Majority Vote: From a PAC-Bayesian Analysis to a Learning Algorithm](https://mlsanity.com/paper/1503.08329/) (2015)
- [Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks](https://mlsanity.com/paper/1905.10259/) (2019)
- [Second Order PAC-Bayesian Bounds for the Weighted Majority Vote](https://mlsanity.com/paper/2007.13532/) (2020)
- [Transductive Rademacher Complexity and its Applications](https://mlsanity.com/paper/1401.3441/) (2014)
- [Rademacher Complexity Bounds for a Penalized Multiclass Semi-Supervised Algorithm](https://mlsanity.com/paper/1607.00567/) (2016)
- [On PAC-Bayesian Bounds for Random Forests](https://mlsanity.com/paper/1810.09746/) (2018)
- [Contrastive Pessimistic Likelihood Estimation for Semi-Supervised Classification](https://mlsanity.com/paper/1503.00269/) (2015)

## Built on by (most cited first, 3 of 3)

- [Self-Training: A Survey](https://mlsanity.com/paper/2202.12040/) (2022)
- [Leveraging Ensemble Diversity for Robust Self-Training in the Presence of Sample Selection Bias](https://mlsanity.com/paper/2310.14814/) (2023)
- [Random Matrix Analysis to Balance between Supervised and Unsupervised Learning under the Low Density Separation Assumption](https://mlsanity.com/paper/2310.13434/) (2023)

## Abstract (arXiv)

Self-learning is a classical approach for learning with both labeled and unlabeled observations which consists in giving pseudo-labels to unlabeled training instances with a confidence score over a predetermined threshold. At the same time, the pseudo-labeling technique is prone to error and runs the risk of adding noisy labels into unlabeled training data. In this paper, we present a probabilistic framework for analyzing self-learning in the multi-class classification scenario with partially labeled data. First, we derive a transductive bound over the risk of the multi-class majority vote classifier. Based on this result, we propose to automatically choose the threshold for pseudo-labeling that minimizes the transductive bound. Then, we introduce a mislabeling error model to analyze the error of the majority vote classifier in the case of the pseudo-labeled data. We derive a probabilistic C-bound over the majority vote error when an imperfect label is given. Empirical results on different data sets show the effectiveness of our framework compared to several state-of-the-art semi-supervised approaches.