# Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning

arXiv: https://arxiv.org/abs/1603.03130 · HTML: https://mlsanity.com/paper/1603.03130/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2016.

## Built on by (most cited first, 30 of 41)

- [Feed Two Birds with One Scone: Exploiting Wild Data for Both Out-of-Distribution Generalization and Detection](https://mlsanity.com/paper/2306.09158/) (2023)
- [Positive-Unlabeled Reward Learning](https://mlsanity.com/paper/1911.00459/) (2019)
- [How Does Unlabeled Data Provably Help Out-of-Distribution Detection?](https://mlsanity.com/paper/2402.03502/) (2024)
- [Meta Discovery: Learning to Discover Novel Classes given Very Limited Data](https://mlsanity.com/paper/2102.04002/) (2021)
- [Learning Bounds for Open-Set Learning](https://mlsanity.com/paper/2106.15792/) (2021)
- [Compete to Win: Enhancing Pseudo Labels for Barely-supervised Medical Image Segmentation](https://mlsanity.com/paper/2304.07519/) (2023)
- [Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practical](https://mlsanity.com/paper/2311.15502/) (2023)
- [Positive Unlabeled Contrastive Learning](https://mlsanity.com/paper/2206.01206/) (2022)
- [Positive-Unlabeled Classification under Class-Prior Shift: A Prior-invariant Approach Based on Density Ratio Estimation](https://mlsanity.com/paper/2107.05045/) (2021)
- [Learning from positive and unlabeled examples -Finite size sample bounds](https://mlsanity.com/paper/2507.07354/) (2025)
- [Unified Risk Analysis for Weakly Supervised Learning](https://mlsanity.com/paper/2309.08216/) (2023)
- [Out-of-Distribution Detection using Synthetic Data Generation](https://mlsanity.com/paper/2502.03323/) (2025)
- [Positive-Unlabeled Reinforcement Learning Distillation for On-Premise Small Models](https://mlsanity.com/paper/2601.20687/) (2026)
- [The Computational Drug Repositioning without Negative Sampling](https://mlsanity.com/paper/2111.14696/) (2021)
- [Unlabeled Imperfect Demonstrations in Adversarial Imitation Learning](https://mlsanity.com/paper/2302.06271/) (2023)
- [Partial-Label Regression](https://mlsanity.com/paper/2306.08968/) (2023)
- [Lambda: Learning Matchable Prior For Entity Alignment with Unlabeled Dangling Cases](https://mlsanity.com/paper/2403.10978/) (2024)
- [PSPU: Enhanced Positive and Unlabeled Learning by Leveraging Pseudo Supervision](https://mlsanity.com/paper/2407.06698/) (2024)
- [Anomaly detection with semi-supervised classification based on risk estimators](https://mlsanity.com/paper/2309.00379/) (2023)
- [Deep Positive-Unlabeled Anomaly Detection for Contaminated Unlabeled Data](https://mlsanity.com/paper/2405.18929/) (2024)
- [Unraveling the Impact of Heterophilic Structures on Graph Positive-Unlabeled Learning](https://mlsanity.com/paper/2405.19919/) (2024)
- [Medix: Out-of-Distribution Detection from Unlabeled Wild Data via Robust Gradient Statistics](https://mlsanity.com/paper/2510.06505/) (2025)
- [Learning from Positive and Unlabeled Data with Augmented Classes](https://mlsanity.com/paper/2207.13274/) (2022)
- [Automatic Debiased Learning from Positive, Unlabeled, and Exposure Data](https://mlsanity.com/paper/2303.04797/) (2023)
- [Accessible, Realistic, and Fair Evaluation of Positive-Unlabeled Learning Algorithms](https://mlsanity.com/paper/2509.24228/) (2025)
- [Semi-Supervised Treatment Effect Estimation with Unlabeled Covariates for Prediction-Powered Causal Inference](https://mlsanity.com/paper/2511.08303/) (2025)
- [A Symmetric Loss Perspective of Reliable Machine Learning](https://mlsanity.com/paper/2101.01366/) (2021)
- [Active Refinement for Multi-Label Learning: A Pseudo-Label Approach](https://mlsanity.com/paper/2109.14676/) (2021)
- [Improving State-of-the-Art in One-Class Classification by Leveraging Unlabeled Data](https://mlsanity.com/paper/2203.07206/) (2022)
- [A Boosting Algorithm for Positive-Unlabeled Learning](https://mlsanity.com/paper/2205.09485/) (2022)

## Abstract (arXiv)

In PU learning, a binary classifier is trained from positive (P) and unlabeled (U) data without negative (N) data. Although N data is missing, it sometimes outperforms PN learning (i.e., ordinary supervised learning). Hitherto, neither theoretical nor experimental analysis has been given to explain this phenomenon. In this paper, we theoretically compare PU (and NU) learning against PN learning based on the upper bounds on estimation errors. We find simple conditions when PU and NU learning are likely to outperform PN learning, and we prove that, in terms of the upper bounds, either PU or NU learning (depending on the class-prior probability and the sizes of P and N data) given infinite U data will improve on PN learning. Our theoretical findings well agree with the experimental results on artificial and benchmark data even when the experimental setup does not match the theoretical assumptions exactly.