# The information-theoretic value of unlabeled data in semi-supervised learning

arXiv: https://arxiv.org/abs/1901.05515 · HTML: https://mlsanity.com/paper/1901.05515/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2019.

## Built on by (most cited first, 5 of 5)

- [Class-Imbalanced Complementary-Label Learning via Weighted Loss](https://mlsanity.com/paper/2209.14189/) (2022)
- [Meta Co-Training: Two Views are Better than One](https://mlsanity.com/paper/2311.18083/) (2023)
- [TRiCo: Triadic Game-Theoretic Co-Training for Robust Semi-Supervised Learning](https://mlsanity.com/paper/2509.21526/) (2025)
- [Learning from Stochastic Labels](https://mlsanity.com/paper/2302.00299/) (2023)
- [Relatively Smart: A New Approach for Instance-Optimal Learning](https://mlsanity.com/paper/2603.01346/) (2026)

## Abstract (arXiv)

We quantify the separation between the numbers of labeled examples required to learn in two settings: Settings with and without the knowledge of the distribution of the unlabeled data. More specifically, we prove a separation by Theta(log n) multiplicative factor for the class of projections over the Boolean hypercube of dimension n. We prove that there is no separation for the class of all functions on domain of any size. Learning with the knowledge of the distribution (a.k.a. fixed-distribution learning) can be viewed as an idealized scenario of semi-supervised learning where the number of unlabeled data points is so great that the unlabeled distribution is known exactly. For this reason, we call the separation the value of unlabeled data.