# Semi-Supervised Learning with Scarce Annotations

arXiv: https://arxiv.org/abs/1905.08845 · HTML: https://mlsanity.com/paper/1905.08845/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2019.

## Built on by (most cited first, 25 of 25)

- [Generalized Category Discovery](https://mlsanity.com/paper/2201.02609/) (2022)
- [AutoNovel: Automatically Discovering and Learning Novel Visual Categories](https://mlsanity.com/paper/2106.15252/) (2021)
- [Parametric Classification for Generalized Category Discovery: A Baseline Study](https://mlsanity.com/paper/2211.11727/) (2022)
- [Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillation](https://mlsanity.com/paper/2107.03358/) (2021)
- [Learning Semi-supervised Gaussian Mixture Models for Generalized Category Discovery](https://mlsanity.com/paper/2305.06144/) (2023)
- [Joint Representation Learning and Novel Category Discovery on Single- and Multi-modal Data](https://mlsanity.com/paper/2104.12673/) (2021)
- [Self-supervised driven consistency training for annotation efficient histopathology image analysis](https://mlsanity.com/paper/2102.03897/) (2021)
- [A Realistic Evaluation of Semi-Supervised Learning for Fine-Grained Classification](https://mlsanity.com/paper/2104.00679/) (2021)
- [Revisiting Consistency Regularization for Semi-Supervised Learning](https://mlsanity.com/paper/2112.05825/) (2021)
- [SelEx: Self-Expertise in Fine-Grained Generalized Category Discovery](https://mlsanity.com/paper/2408.14371/) (2024)
- [Reference-guided Pseudo-Label Generation for Medical Semantic Segmentation](https://mlsanity.com/paper/2112.00735/) (2021)
- [OpenCon: Open-world Contrastive Learning](https://mlsanity.com/paper/2208.02764/) (2022)
- [NBIAS: A Natural Language Processing Framework for Bias Identification in Text](https://mlsanity.com/paper/2308.01681/) (2023)
- [A Graph-Theoretic Framework for Understanding Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2311.03524/) (2023)
- [Enhancing Modality-Agnostic Representations via Meta-Learning for Brain Tumor Segmentation](https://mlsanity.com/paper/2302.04308/) (2023)
- [Dynamic Curriculum Learning for Great Ape Detection in the Wild](https://mlsanity.com/paper/2205.00275/) (2022)
- [RoPAWS: Robust Semi-supervised Representation Learning from Uncurated Data](https://mlsanity.com/paper/2302.14483/) (2023)
- [Semi-supervised classification of radiology images with NoTeacher: A Teacher that is not Mean](https://mlsanity.com/paper/2108.04423/) (2021)
- [Semi-supervised Long-tailed Recognition using Alternate Sampling](https://mlsanity.com/paper/2105.00133/) (2021)
- [Distribution Estimation to Automate Transformation Policies for Self-Supervision](https://mlsanity.com/paper/2111.12265/) (2021)
- [A Decade of You Only Look Once (YOLO) for Object Detection: A Review](https://mlsanity.com/paper/2504.18586/) (2025)
- [Memory Consistency Guided Divide-and-Conquer Learning for Generalized Category Discovery](https://mlsanity.com/paper/2401.13325/) (2024)
- [Generalized Category Discovery under the Long-Tailed Distribution](https://mlsanity.com/paper/2506.12515/) (2025)
- [Balancing Uncertainty and Diversity of Samples: Leveraging Diversity of Least, High Confidence Samples for Effective Active Learning](https://mlsanity.com/paper/2605.22169/) (2026)
- [Fourier Self-Supervision for Fine-Grained Generalized Category Discovery](https://mlsanity.com/paper/2608.08963/) (2026)

## Abstract (arXiv)

While semi-supervised learning (SSL) algorithms provide an efficient way to make use of both labelled and unlabelled data, they generally struggle when the number of annotated samples is very small. In this work, we consider the problem of SSL multi-class classification with very few labelled instances. We introduce two key ideas. The first is a simple but effective one: we leverage the power of transfer learning among different tasks and self-supervision to initialize a good representation of the data without making use of any label. The second idea is a new algorithm for SSL that can exploit well such a pre-trained representation. The algorithm works by alternating two phases, one fitting the labelled points and one fitting the unlabelled ones, with carefully-controlled information flow between them. The benefits are greatly reducing overfitting of the labelled data and avoiding issue with balancing labelled and unlabelled losses during training. We show empirically that this method can successfully train competitive models with as few as 10 labelled data points per class. More in general, we show that the idea of bootstrapping features using self-supervised learning always improves SSL on standard benchmarks. We show that our algorithm works increasingly well compared to other methods when refining from other tasks or datasets.