# Split-PU: Hardness-aware Training Strategy for Positive-Unlabeled Learning

arXiv: https://arxiv.org/abs/2211.16756 · HTML: https://mlsanity.com/paper/2211.16756/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 30 Nov 2022.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [Joint Optimization Framework for Learning with Noisy Labels](https://mlsanity.com/paper/1803.11364/) (2018)
- [Interpolation Consistency Training for Semi-Supervised Learning](https://mlsanity.com/paper/1903.03825/) (2019)
- [Positive-Unlabeled Learning with Non-Negative Risk Estimator](https://mlsanity.com/paper/1703.00593/) (2017)
- [Learning from Noisy Labels with Distillation](https://mlsanity.com/paper/1703.02391/) (2017)
- [Webly Supervised Learning of Convolutional Networks](https://mlsanity.com/paper/1505.01554/) (2015)
- [Toward Robustness against Label Noise in Training Deep Discriminative Neural Networks](https://mlsanity.com/paper/1706.00038/) (2017)
- [Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels](https://mlsanity.com/paper/2105.04522/) (2021)
- [Self-PU: Self Boosted and Calibrated Positive-Unlabeled Training](https://mlsanity.com/paper/2006.11280/) (2020)
- [Coherent Gradients: An Approach to Understanding Generalization in Gradient Descent-based Optimization](https://mlsanity.com/paper/2002.10657/) (2020)
- [Classification from Positive, Unlabeled and Biased Negative Data](https://mlsanity.com/paper/1810.00846/) (2018)
- [Deep Saliency with Encoded Low level Distance Map and High Level Features](https://mlsanity.com/paper/1604.05495/) (2016)
- [Scalable Penalized Regression for Noise Detection in Learning with Noisy Labels](https://mlsanity.com/paper/2203.07788/) (2022)
- [Revisiting Sample Selection Approach to Positive-Unlabeled Learning: Turning Unlabeled Data into Positive rather than Negative](https://mlsanity.com/paper/1901.10155/) (2019)
- [Uncertainty-aware Pseudo-label Selection for Positive-Unlabeled Learning](https://mlsanity.com/paper/2201.13192/) (2022)
- [A Variational Approach for Learning from Positive and Unlabeled Data](https://mlsanity.com/paper/1906.00642/) (2019)

## Built on by (most cited first, 2 of 2)

- [PSPU: Enhanced Positive and Unlabeled Learning by Leveraging Pseudo Supervision](https://mlsanity.com/paper/2407.06698/) (2024)
- [PUAL: A Classifier on Trifurcate Positive-Unlabeled Data](https://mlsanity.com/paper/2405.20970/) (2024)

## Abstract (arXiv)

Positive-Unlabeled (PU) learning aims to learn a model with rare positive samples and abundant unlabeled samples. Compared with classical binary classification, the task of PU learning is much more challenging due to the existence of many incompletely-annotated data instances. Since only part of the most confident positive samples are available and evidence is not enough to categorize the rest samples, many of these unlabeled data may also be the positive samples. Research on this topic is particularly useful and essential to many real-world tasks which demand very expensive labelling cost. For example, the recognition tasks in disease diagnosis, recommendation system and satellite image recognition may only have few positive samples that can be annotated by the experts. These methods mainly omit the intrinsic hardness of some unlabeled data, which can result in sub-optimal performance as a consequence of fitting the easy noisy data and not sufficiently utilizing the hard data. In this paper, we focus on improving the commonly-used nnPU with a novel training pipeline. We highlight the intrinsic difference of hardness of samples in the dataset and the proper learning strategies for easy and hard data. By considering this fact, we propose first splitting the unlabeled dataset with an early-stop strategy. The samples that have inconsistent predictions between the temporary and base model are considered as hard samples. Then the model utilizes a noise-tolerant Jensen-Shannon divergence loss for easy data; and a dual-source consistency regularization for hard data which includes a cross-consistency between student and base model for low-level features and self-consistency for high-level features and predictions, respectively.