# An Apparent Paradox: A Classifier Trained from a Partially Classified Sample May Have Smaller Expected Error Rate Than That If the Sample Were Completely Classified

arXiv: https://arxiv.org/abs/1910.09189 · HTML: https://mlsanity.com/paper/1910.09189/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2019.

## Built on by (most cited first, 8 of 8)

- [A Survey on Semi-Supervised Learning for Delayed Partially Labelled Data Streams](https://mlsanity.com/paper/2106.09170/) (2021)
- [Semi-Supervised Learning of Classifiers from a Statistical Perspective: A Brief Review](https://mlsanity.com/paper/2104.04046/) (2021)
- [Analysis of Estimating the Bayes Rule for Gaussian Mixture Models with a Specified Missing-Data Mechanism](https://mlsanity.com/paper/2210.13785/) (2022)
- [Informative missingness and its implications in semi-supervised learning](https://mlsanity.com/paper/2512.04392/) (2025)
- [Learning from Uncertainty-dependent Missing Labels for Semi-supervised Classification](https://mlsanity.com/paper/2608.23960/) (2026)
- [SSLfmm: An R Package for Semi-Supervised Learning with Mixed Missingness](https://mlsanity.com/paper/2512.03322/) (2025)
- [Semi-Supervised Mixture Models under the Concept of Missing at Radom with Margin Confidence and Aranda Ordaz Function](https://mlsanity.com/paper/2601.14631/) (2026)
- [Semi-Supervised Classification with Informative Missing Labels in Weibull Mixture Models](https://mlsanity.com/paper/2609.00774/) (2026)

## Abstract (arXiv)

There has been increasing interest in using semi-supervised learning to form a classifier. As is well known, the (Fisher) information in an unclassified feature with unknown class label is less (considerably less for weakly separated classes) than that of a classified feature which has known class label. Hence assuming that the labels of the unclassified features are randomly missing or their missing-label mechanism is simply ignored, the expected error rate of a classifier formed from a partially classified sample is greater than that if the sample were completely classified. We propose to treat the labels of the unclassified features as missing data and to introduce a framework for their missingness in situations where these labels are not randomly missing. An examination of several partially classified data sets in the literature suggests that the unclassified features are not occurring at random but rather tend to be concentrated in regions of relatively high entropy in the feature space. Here in the context of two normal classes with a common covariance matrix we consider the situation where the missingness of the labels of the unclassified features can be modelled by a logistic model in which the probability of a missing label for a feature depends on its entropy. Rather paradoxically, we show that the classifier so formed from the partially classified sample may have smaller expected error rate that that if the sample were completely classified.