# SemiOccam: A Robust Semi-Supervised Image Recognition Network Using Sparse Labels

arXiv: https://arxiv.org/abs/2506.03582 · HTML: https://mlsanity.com/paper/2506.03582/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 37 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [Deep Layer Aggregation](https://mlsanity.com/paper/1707.06484/) (2017)
- [Understanding Dimensional Collapse in Contrastive Self-supervised Learning](https://mlsanity.com/paper/2110.09348/) (2021)
- [Meta Pseudo Labels](https://mlsanity.com/paper/2003.10580/) (2020)
- [FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning](https://mlsanity.com/paper/2205.07246/) (2022)
- [Generalized Category Discovery](https://mlsanity.com/paper/2201.02609/) (2022)
- [CoMatch: Semi-supervised Learning with Contrastive Graph Regularization](https://mlsanity.com/paper/2011.11183/) (2020)
- [Dash: Semi-Supervised Learning with Dynamic Thresholding](https://mlsanity.com/paper/2109.00650/) (2021)
- [SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learning](https://mlsanity.com/paper/2301.10921/) (2023)
- [SimMatch: Semi-supervised Learning with Similarity Matching](https://mlsanity.com/paper/2203.06915/) (2022)
- [AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation](https://mlsanity.com/paper/2106.04732/) (2021)
- [USB: A Unified Semi-supervised Learning Benchmark for Classification](https://mlsanity.com/paper/2208.07204/) (2022)
- [Learning Semi-supervised Gaussian Mixture Models for Generalized Category Discovery](https://mlsanity.com/paper/2305.06144/) (2023)
- [S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captions](https://mlsanity.com/paper/2305.14095/) (2023)
- [Intrinsic Self-Supervision for Data Quality Audits](https://mlsanity.com/paper/2305.17048/) (2023)
- [SequenceMatch: Revisiting the design of weak-strong augmentations for Semi-supervised learning](https://mlsanity.com/paper/2310.15787/) (2023)
- [The Role of Pseudo-labels in Self-training Linear Classifiers on High-dimensional Gaussian Mixture Data](https://mlsanity.com/paper/2205.07739/) (2022)
- [Performance of Gaussian Mixture Model Classifiers on Embedded Feature Spaces](https://mlsanity.com/paper/2410.13421/) (2024)
- [Debiasing, calibrating, and improving Semi-supervised Learning performance via simple Ensemble Projector](https://mlsanity.com/paper/2310.15764/) (2023)
- [Better (pseudo-)labels for semi-supervised instance segmentation](https://mlsanity.com/paper/2403.11675/) (2024)

## Abstract (arXiv)

We present SemiOccam, an image recognition network that leverages semi-supervised learning in a highly efficient manner. Existing works often rely on complex training techniques and architectures, requiring hundreds of GPU hours for training, while their generalization ability with extremely limited labeled data remains to be improved. To address these limitations, we construct a hierarchical mixture density classification mechanism by optimizing mutual information between feature representations and target classes, compressing redundant information while retaining crucial discriminative components. Experimental results demonstrate that our method achieves state-of-the-art performance on three commonly used datasets, with accuracy exceeding 95% on two of them using only 4 labeled samples per class, and its simple architecture keeps training time at the minute level. Notably, this paper reveals a long-overlooked data leakage issue in the STL-10 dataset for semi-supervised learning and removes duplicates to ensure reliable experimental results. We release the deduplicated CleanSTL-10 dataset to facilitate fair and reproducible research. Code available at https://github.com/Shu1L0n9/SemiOccam.