# Combining Self-labeling with Selective Sampling

arXiv: https://arxiv.org/abs/2301.04420 · HTML: https://mlsanity.com/paper/2301.04420/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 59 references that are in the map.
- First posted on arXiv 11 Jan 2023.

## Builds on (most cited first, 20 of 20)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [Self-labelling via simultaneous clustering and representation learning](https://mlsanity.com/paper/1911.05371/) (2019)
- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [Cost-Effective Active Learning for Deep Image Classification](https://mlsanity.com/paper/1701.03551/) (2017)
- [Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks](https://mlsanity.com/paper/1903.11680/) (2019)
- [Deep Self-Learning From Noisy Labels](https://mlsanity.com/paper/1908.02160/) (2019)
- [Semi-Supervised Active Learning with Temporal Output Discrepancy](https://mlsanity.com/paper/2107.14153/) (2021)
- [Rethinking deep active learning: Using unlabeled data at model training](https://mlsanity.com/paper/1911.08177/) (2019)
- [Label Noise Types and Their Effects on Deep Learning](https://mlsanity.com/paper/2003.10471/) (2020)
- [Is margin all you need? An extensive empirical study of active learning on tabular data](https://mlsanity.com/paper/2210.03822/) (2022)
- [Minimum-Margin Active Learning](https://mlsanity.com/paper/1906.00025/) (2019)
- [Exploiting Diversity of Unlabeled Data for Label-Efficient Semi-Supervised Active Learning](https://mlsanity.com/paper/2207.12302/) (2022)
- [Mining Drifting Data Streams on a Budget: Combining Active Learning with Self-Labeling](https://mlsanity.com/paper/2112.11019/) (2021)

## Built on by (most cited first, 1 of 1)

- [Cross-Modality Clustering-based Self-Labeling for Multimodal Data Classification](https://mlsanity.com/paper/2408.02568/) (2024)

## Abstract (arXiv)

Since data is the fuel that drives machine learning models, and access to labeled data is generally expensive, semi-supervised methods are constantly popular. They enable the acquisition of large datasets without the need for too many expert labels. This work combines self-labeling techniques with active learning in a selective sampling scenario. We propose a new method that builds an ensemble classifier. Based on an evaluation of the inconsistency of the decisions of the individual base classifiers for a given observation, a decision is made on whether to request a new label or use the self-labeling. In preliminary studies, we show that naive application of self-labeling can harm performance by introducing bias towards selected classes and consequently lead to skewed class distribution. Hence, we also propose mechanisms to reduce this phenomenon. Experimental evaluation shows that the proposed method matches current selective sampling methods or achieves better results.