# Meta-Semi: A Meta-learning Approach for Semi-supervised Learning

arXiv: https://arxiv.org/abs/2007.02394 · HTML: https://mlsanity.com/paper/2007.02394/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2020.

## Built on by (most cited first, 11 of 11)

- [Revisiting Locally Supervised Learning: an Alternative to End-to-end Training](https://mlsanity.com/paper/2101.10832/) (2021)
- [Glance and Focus Networks for Dynamic Visual Recognition](https://mlsanity.com/paper/2201.03014/) (2022)
- [Betty: An Automatic Differentiation Library for Multilevel Optimization](https://mlsanity.com/paper/2207.02849/) (2022)
- [A Review of Pseudo-Labeling for Computer Vision](https://mlsanity.com/paper/2408.07221/) (2024)
- [Semi-Supervised Learning of Classifiers from a Statistical Perspective: A Brief Review](https://mlsanity.com/paper/2104.04046/) (2021)
- [Meta-learning of semi-supervised learning from tasks with heterogeneous attribute spaces](https://mlsanity.com/paper/2311.05088/) (2023)
- [Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts](https://mlsanity.com/paper/2403.12918/) (2024)
- [Reinforcement Learning-Guided Semi-Supervised Learning](https://mlsanity.com/paper/2405.01760/) (2024)
- [On the Difficulty of Learning a Meta-network for Training Data Selection](https://mlsanity.com/paper/2606.00571/) (2026)
- [Learning from Mistakes based on Class Weighting with Application to Neural Architecture Search](https://mlsanity.com/paper/2112.00275/) (2021)
- [Scalable, Transferable Meta-network for Data Selection Requires a Different Loss (and Why the Obvious Choice is Problematic)](https://mlsanity.com/paper/2610.02092/) (2026)

## Abstract (arXiv)

Deep learning based semi-supervised learning (SSL) algorithms have led to promising results in recent years. However, they tend to introduce multiple tunable hyper-parameters, making them less practical in real SSL scenarios where the labeled data is scarce for extensive hyper-parameter search. In this paper, we propose a novel meta-learning based SSL algorithm (Meta-Semi) that requires tuning only one additional hyper-parameter, compared with a standard supervised deep learning algorithm, to achieve competitive performance under various conditions of SSL. We start by defining a meta optimization problem that minimizes the loss on labeled data through dynamically reweighting the loss on unlabeled samples, which are associated with soft pseudo labels during training. As the meta problem is computationally intensive to solve directly, we propose an efficient algorithm to dynamically obtain the approximate solutions. We show theoretically that Meta-Semi converges to the stationary point of the loss function on labeled data under mild conditions. Empirically, Meta-Semi outperforms state-of-the-art SSL algorithms significantly on the challenging semi-supervised CIFAR-100 and STL-10 tasks, and achieves competitive performance on CIFAR-10 and SVHN.