# Enhancing Semi-Supervised Learning via Representative and Diverse Sample Selection

arXiv: https://arxiv.org/abs/2409.11653 · HTML: https://mlsanity.com/paper/2409.11653/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 68 references that are in the map.
- First posted on arXiv 18 Sep 2024.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning](https://mlsanity.com/paper/1606.04586/) (2016)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning](https://mlsanity.com/paper/2205.07246/) (2022)
- [Cost-Effective Active Learning for Deep Image Classification](https://mlsanity.com/paper/1701.03551/) (2017)
- [Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Cost](https://mlsanity.com/paper/1910.07153/) (2019)
- [USB: A Unified Semi-supervised Learning Benchmark for Classification](https://mlsanity.com/paper/2208.07204/) (2022)
- [Debiased Learning from Naturally Imbalanced Pseudo-Labels](https://mlsanity.com/paper/2201.01490/) (2022)
- [Influence Selection for Active Learning](https://mlsanity.com/paper/2108.09331/) (2021)
- [Semi-Supervised Active Learning with Temporal Output Discrepancy](https://mlsanity.com/paper/2107.14153/) (2021)
- [Support points](https://mlsanity.com/paper/1609.01811/) (2016)
- [Active Finetuning: Exploiting Annotation Budget in the Pretraining-Finetuning Paradigm](https://mlsanity.com/paper/2303.14382/) (2023)
- [Low Budget Active Learning via Wasserstein Distance: An Integer Programming Approach](https://mlsanity.com/paper/2106.02968/) (2021)
- [Unsupervised Selective Labeling for More Effective Semi-Supervised Learning](https://mlsanity.com/paper/2110.03006/) (2021)
- [Concentration Inequalities for Statistical Inference](https://mlsanity.com/paper/2011.02258/) (2020)
- [On the Marginal Benefit of Active Learning: Does Self-Supervision Eat Its Cake?](https://mlsanity.com/paper/2011.08121/) (2020)
- [MCDAL: Maximum Classifier Discrepancy for Active Learning](https://mlsanity.com/paper/2107.11049/) (2021)
- [Combining MixMatch and Active Learning for Better Accuracy with Fewer Labels](https://mlsanity.com/paper/1912.00594/) (2019)
- [Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learning](https://mlsanity.com/paper/2308.06777/) (2023)
- [Optimal Distributed Subsampling for Maximum Quasi-Likelihood Estimators with Massive Data](https://mlsanity.com/paper/2005.10435/) (2020)
- [InstanT: Semi-supervised Learning with Instance-dependent Thresholds](https://mlsanity.com/paper/2310.18910/) (2023)

## Built on by (most cited first, 2 of 2)

- [LUCoS: Latent Unsupervised Context Selection for Tabular Foundation Models](https://mlsanity.com/paper/2605.27254/) (2026)
- [Towards Clinical Practice in CT-Based Pulmonary Disease Screening: An Efficient and Reliable Framework](https://mlsanity.com/paper/2412.01525/) (2024)

## Abstract (arXiv)

Semi-Supervised Learning (SSL) has become a preferred paradigm in many deep learning tasks, which reduces the need for human labor. Previous studies primarily focus on effectively utilising the labelled and unlabeled data to improve performance. However, we observe that how to select samples for labelling also significantly impacts performance, particularly under extremely low-budget settings. The sample selection task in SSL has been under-explored for a long time. To fill in this gap, we propose a Representative and Diverse Sample Selection approach (RDSS). By adopting a modified Frank-Wolfe algorithm to minimise a novel criterion α-Maximum Mean Discrepancy (α-MMD), RDSS samples a representative and diverse subset for annotation from the unlabeled data. We demonstrate that minimizing α-MMD enhances the generalization ability of low-budget learning. Experimental results show that RDSS consistently improves the performance of several popular SSL frameworks and outperforms the state-of-the-art sample selection approaches used in Active Learning (AL) and Semi-Supervised Active Learning (SSAL), even with constrained annotation budgets.