# Mitigating Overfitting in Supervised Classification from Two Unlabeled Datasets: A Consistent Risk Correction Approach

arXiv: https://arxiv.org/abs/1910.08974 · HTML: https://mlsanity.com/paper/1910.08974/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2019.

## Built on by (most cited first, 29 of 29)

- [Learning from Similarity-Confidence Data](https://mlsanity.com/paper/2102.06879/) (2021)
- [Binary Classification with Confidence Difference](https://mlsanity.com/paper/2310.05632/) (2023)
- [Binary Classification from Multiple Unlabeled Datasets via Surrogate Set Classification](https://mlsanity.com/paper/2102.00678/) (2021)
- [Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practical](https://mlsanity.com/paper/2311.15502/) (2023)
- [Positive-Unlabeled Classification under Class-Prior Shift: A Prior-invariant Approach Based on Density Ratio Estimation](https://mlsanity.com/paper/2107.05045/) (2021)
- [Multi-class Classification from Multiple Unlabeled Datasets with Partial Risk Regularization](https://mlsanity.com/paper/2207.01555/) (2022)
- [Federated Learning under Covariate Shifts with Generalization Guarantees](https://mlsanity.com/paper/2306.05325/) (2023)
- [Unified Risk Analysis for Weakly Supervised Learning](https://mlsanity.com/paper/2309.08216/) (2023)
- [Making Binary Classification from Multiple Unlabeled Datasets Almost Free of Supervision](https://mlsanity.com/paper/2306.07036/) (2023)
- [On Symmetric Losses for Robust Policy Optimization with Noisy Preferences](https://mlsanity.com/paper/2505.24709/) (2025)
- [Multi-Class Classification from Single-Class Data with Confidences](https://mlsanity.com/paper/2106.08864/) (2021)
- [Rethinking Consistent Multi-Label Classification Under Inexact Supervision](https://mlsanity.com/paper/2510.04091/) (2025)
- [Learning from Label Proportions with Instance-wise Consistency](https://mlsanity.com/paper/2203.12836/) (2022)
- [A Generalized Unbiased Risk Estimator for Learning with Augmented Classes](https://mlsanity.com/paper/2306.06894/) (2023)
- [Self Iterative Label Refinement via Robust Unlabeled Learning](https://mlsanity.com/paper/2502.12565/) (2025)
- [Lower-Bounded Proper Losses for Weakly Supervised Classification](https://mlsanity.com/paper/2103.02893/) (2021)
- [Automatic Debiased Learning from Positive, Unlabeled, and Exposure Data](https://mlsanity.com/paper/2303.04797/) (2023)
- [A Symmetric Loss Perspective of Reliable Machine Learning](https://mlsanity.com/paper/2101.01366/) (2021)
- [Enhancing Affinity Propagation for Improved Public Sentiment Insights](https://mlsanity.com/paper/2410.12862/) (2024)
- [Uncertainty-aware Long-tailed Weights Model the Utility of Pseudo-labels for Semi-supervised Learning](https://mlsanity.com/paper/2503.09974/) (2025)
- [Reduction of Supervision for Biomedical Knowledge Discovery](https://mlsanity.com/paper/2504.09582/) (2025)
- [Learning from M-Tuple Dominant Positive and Unlabeled Data](https://mlsanity.com/paper/2506.15686/) (2025)
- [A Unified Empirical Risk Minimization Framework for Flexible N-Tuples Weak Supervision](https://mlsanity.com/paper/2507.07771/) (2025)
- [Learning from Uncertain Similarity and Unlabeled Data](https://mlsanity.com/paper/2509.11984/) (2025)
- [Learning from N-Tuple Data with M Positive Instances: Unbiased Risk Estimation and Theoretical Guarantees](https://mlsanity.com/paper/2510.18406/) (2025)
- [A Unified and Stable Risk Minimization Framework for Weakly Supervised Learning with Theoretical Guarantees](https://mlsanity.com/paper/2511.22823/) (2025)
- [Generalized Distribution-Free Semi-Supervised Learning with Risk Rewrite](https://mlsanity.com/paper/2607.11947/) (2026)
- [Recent advances in weakly supervised learning: New supervision paradigms, assumption relaxations, and practical solutions](https://mlsanity.com/paper/2608.06896/) (2026)
- [Importance Weighting for Unlabeled-unlabeled Learning under Distribution Shift](https://mlsanity.com/paper/2609.10994/) (2026)

## Abstract (arXiv)

The recently proposed unlabeled-unlabeled (UU) classification method allows us to train a binary classifier only from two unlabeled datasets with different class priors. Since this method is based on the empirical risk minimization, it works as if it is a supervised classification method, compatible with any model and optimizer. However, this method sometimes suffers from severe overfitting, which we would like to prevent in this paper. Our empirical finding in applying the original UU method is that overfitting often co-occurs with the empirical risk going negative, which is not legitimate. Therefore, we propose to wrap the terms that cause a negative empirical risk by certain correction functions. Then, we prove the consistency of the corrected risk estimator and derive an estimation error bound for the corrected risk minimizer. Experiments show that our proposal can successfully mitigate overfitting of the UU method and significantly improve the classification accuracy.