# Semi-Supervised Learning with Meta-Gradient

arXiv: https://arxiv.org/abs/2007.03966 · HTML: https://mlsanity.com/paper/2007.03966/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jul 2020.

## Built on by (most cited first, 7 of 7)

- [On Training Implicit Models](https://mlsanity.com/paper/2111.05177/) (2021)
- [Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradients](https://mlsanity.com/paper/2202.08132/) (2022)
- [Learning to Predict Gradients for Semi-Supervised Continual Learning](https://mlsanity.com/paper/2201.09196/) (2022)
- [Confident Sinkhorn Allocation for Pseudo-Labeling](https://mlsanity.com/paper/2206.05880/) (2022)
- [Meta-learning of semi-supervised learning from tasks with heterogeneous attribute spaces](https://mlsanity.com/paper/2311.05088/) (2023)
- [CLID-MU: Cross-Layer Information Divergence Based Meta Update Strategy for Learning with Noisy Labels](https://mlsanity.com/paper/2507.11807/) (2025)
- [MTSSL: Meta-Thresholding Semi-Supervised Learning](https://mlsanity.com/paper/2607.16363/) (2026)

## Abstract (arXiv)

In this work, we propose a simple yet effective meta-learning algorithm in semi-supervised learning. We notice that most existing consistency-based approaches suffer from overfitting and limited model generalization ability, especially when training with only a small number of labeled data. To alleviate this issue, we propose a learn-to-generalize regularization term by utilizing the label information and optimize the problem in a meta-learning fashion. Specifically, we seek the pseudo labels of the unlabeled data so that the model can generalize well on the labeled data, which is formulated as a nested optimization problem. We address this problem using the meta-gradient that bridges between the pseudo label and the regularization term. In addition, we introduce a simple first-order approximation to avoid computing higher-order derivatives and provide theoretic convergence analysis. Extensive evaluations on the SVHN, CIFAR, and ImageNet datasets demonstrate that the proposed algorithm performs favorably against state-of-the-art methods.