# Generalized Distribution-Free Semi-Supervised Learning with Risk Rewrite

arXiv: https://arxiv.org/abs/2607.11947 · HTML: https://mlsanity.com/paper/2607.11947/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 11 Jul 2026.

## Builds on (most cited first, 28 of 28)

- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Meta Pseudo Labels](https://mlsanity.com/paper/2003.10580/) (2020)
- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning](https://mlsanity.com/paper/2205.07246/) (2022)
- [Positive-Unlabeled Learning with Non-Negative Risk Estimator](https://mlsanity.com/paper/1703.00593/) (2017)
- [An Overview of Deep Semi-Supervised Learning](https://mlsanity.com/paper/2006.05278/) (2020)
- [Learning from Complementary Labels](https://mlsanity.com/paper/1705.07541/) (2017)
- [Self-Training: A Survey](https://mlsanity.com/paper/2202.12040/) (2022)
- [Debiased Learning from Naturally Imbalanced Pseudo-Labels](https://mlsanity.com/paper/2201.01490/) (2022)
- [Mixture Proportion Estimation via Kernel Embedding of Distributions](https://mlsanity.com/paper/1603.02501/) (2016)
- [On Symmetric Losses for Learning from Corrupted Labels](https://mlsanity.com/paper/1901.09314/) (2019)
- [Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning](https://mlsanity.com/paper/1603.03130/) (2016)
- [Mitigating Overfitting in Supervised Classification from Two Unlabeled Datasets: A Consistent Risk Correction Approach](https://mlsanity.com/paper/1910.08974/) (2019)
- [Learning from Positive and Unlabeled Data with Arbitrary Positive Shift](https://mlsanity.com/paper/2002.10261/) (2020)
- [Classification from Pairwise Similarities/Dissimilarities and Unlabeled Data via Empirical Risk Minimization](https://mlsanity.com/paper/1904.11717/) (2019)
- [Semi-Supervised AUC Optimization based on Positive-Unlabeled Learning](https://mlsanity.com/paper/1705.01708/) (2017)
- [Kernel Density Estimation for Multiclass Quantification](https://mlsanity.com/paper/2401.00490/) (2023)
- [Learning Classifiers under Delayed Feedback with a Time Window Assumption](https://mlsanity.com/paper/2009.13092/) (2020)
- [Multi-class Classification from Multiple Unlabeled Datasets with Partial Risk Regularization](https://mlsanity.com/paper/2207.01555/) (2022)
- [Anomaly detection with semi-supervised classification based on risk estimators](https://mlsanity.com/paper/2309.00379/) (2023)
- [Multi-Agent VLMs Guided Self-Training with PNU Loss for Low-Resource Offensive Content Detection](https://mlsanity.com/paper/2511.13759/) (2025)
- [Robust Semi-supervised Least Squares Classification by Implicit Constraints](https://mlsanity.com/paper/1512.08240/) (2015)
- [Binary Matrix Completion Using Unobserved Entries](https://mlsanity.com/paper/1803.04663/) (2018)
- [Semi-Supervised Ordinal Regression Based on Empirical Risk Minimization](https://mlsanity.com/paper/1901.11351/) (2019)

## Abstract (arXiv)

Typical semi-supervised learning (SSL) methods rely on distributional assumptions, and their performance degrades when these are violated. While PNU learning, a risk rewriting method, offers a distribution-free alternative, it is restricted to binary classification and its variance optimality remains unclear. In this paper, we propose a generalized framework that constructs unbiased risk estimators using linear combinations of component risks, subsuming PNU learning and extending to multiclass classification. We derive the minimum achievable variance, demonstrating our estimator can attain lower variance than PNU in asymmetric loss scenarios. Furthermore, we establish a generalization bound directly linking this variance reduction to improved learning performance. Based on these theoretical insights, we introduce two practical SSL methods that empirically match or outperform existing approaches on binary and multiclass benchmarks.