# S4L: Self-Supervised Semi-Supervised Learning

arXiv: https://arxiv.org/abs/1905.03670 · HTML: https://mlsanity.com/paper/1905.03670/

## Key facts

- Cited by 335 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2019.

## Built on by (most cited first, 30 of 335)

- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases](https://mlsanity.com/paper/2103.10697/) (2021)
- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [A Survey on Deep Semi-supervised Learning](https://mlsanity.com/paper/2103.00550/) (2021)
- [FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning](https://mlsanity.com/paper/2205.07246/) (2022)
- [Generalized Category Discovery](https://mlsanity.com/paper/2201.02609/) (2022)
- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2106.05095/) (2021)
- [Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2102.03526/) (2021)
- [SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption](https://mlsanity.com/paper/2106.15147/) (2021)
- [Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization](https://mlsanity.com/paper/2104.05833/) (2021)
- [SEED: Self-supervised Distillation For Visual Representation](https://mlsanity.com/paper/2101.04731/) (2021)
- [SimMatch: Semi-supervised Learning with Similarity Matching](https://mlsanity.com/paper/2203.06915/) (2022)
- [Humble Teachers Teach Better Students for Semi-Supervised Object Detection](https://mlsanity.com/paper/2106.10456/) (2021)
- [AutoNovel: Automatically Discovering and Learning Novel Visual Categories](https://mlsanity.com/paper/2106.15252/) (2021)
- [Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples](https://mlsanity.com/paper/2104.13963/) (2021)
- [Parametric Classification for Generalized Category Discovery: A Baseline Study](https://mlsanity.com/paper/2211.11727/) (2022)
- [Semi-supervised Semantic Segmentation with Directional Context-aware Consistency](https://mlsanity.com/paper/2106.14133/) (2021)
- [3D Human Action Representation Learning via Cross-View Consistency Pursuit](https://mlsanity.com/paper/2104.14466/) (2021)
- [Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2108.09025/) (2021)
- [USB: A Unified Semi-supervised Learning Benchmark for Classification](https://mlsanity.com/paper/2208.07204/) (2022)
- [Dynamic Conceptional Contrastive Learning for Generalized Category Discovery](https://mlsanity.com/paper/2303.17393/) (2023)
- [Guided Point Contrastive Learning for Semi-supervised Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2110.08188/) (2021)
- [PromptCAL: Contrastive Affinity Learning via Auxiliary Prompts for Generalized Novel Category Discovery](https://mlsanity.com/paper/2212.05590/) (2022)
- [Divide and Contrast: Self-supervised Learning from Uncurated Data](https://mlsanity.com/paper/2105.08054/) (2021)
- [Weakly Supervised Contrastive Learning](https://mlsanity.com/paper/2110.04770/) (2021)
- [PixMatch: Unsupervised Domain Adaptation via Pixelwise Consistency Training](https://mlsanity.com/paper/2105.08128/) (2021)
- [Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning](https://mlsanity.com/paper/2101.08482/) (2021)
- [A Broad Study on the Transferability of Visual Representations with Contrastive Learning](https://mlsanity.com/paper/2103.13517/) (2021)

## Abstract (arXiv)

This work tackles the problem of semi-supervised learning of image classifiers. Our main insight is that the field of semi-supervised learning can benefit from the quickly advancing field of self-supervised visual representation learning. Unifying these two approaches, we propose the framework of self-supervised semi-supervised learning and use it to derive two novel semi-supervised image classification methods. We demonstrate the effectiveness of these methods in comparison to both carefully tuned baselines, and existing semi-supervised learning methods. We then show that our approach and existing semi-supervised methods can be jointly trained, yielding a new state-of-the-art result on semi-supervised ILSVRC-2012 with 10% of labels.