# Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning

arXiv: https://arxiv.org/abs/2007.01293 · HTML: https://mlsanity.com/paper/2007.01293/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2020.

## Built on by (most cited first, 30 of 53)

- [A Survey on Deep Semi-supervised Learning](https://mlsanity.com/paper/2103.00550/) (2021)
- [A Survey of Human-in-the-loop for Machine Learning](https://mlsanity.com/paper/2108.00941/) (2021)
- [ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2106.05095/) (2021)
- [Dash: Semi-Supervised Learning with Dynamic Thresholding](https://mlsanity.com/paper/2109.00650/) (2021)
- [SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learning](https://mlsanity.com/paper/2301.10921/) (2023)
- [Influence Selection for Active Learning](https://mlsanity.com/paper/2108.09331/) (2021)
- [Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation](https://mlsanity.com/paper/2207.14191/) (2022)
- [DASO: Distribution-Aware Semantics-Oriented Pseudo-label for Imbalanced Semi-Supervised Learning](https://mlsanity.com/paper/2106.05682/) (2021)
- [Unbiased Subclass Regularization for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.10026/) (2022)
- [In Defense of Core-set: A Density-aware Core-set Selection for Active Learning](https://mlsanity.com/paper/2206.04838/) (2022)
- [Betty: An Automatic Differentiation Library for Multilevel Optimization](https://mlsanity.com/paper/2207.02849/) (2022)
- [Equivariance Discovery by Learned Parameter-Sharing](https://mlsanity.com/paper/2204.03640/) (2022)
- [IMMA: Immunizing text-to-image Models against Malicious Adaptation](https://mlsanity.com/paper/2311.18815/) (2023)
- [FairIF: Boosting Fairness in Deep Learning via Influence Functions with Validation Set Sensitive Attributes](https://mlsanity.com/paper/2201.05759/) (2022)
- [An Embarrassingly Simple Baseline for Imbalanced Semi-Supervised Learning](https://mlsanity.com/paper/2211.11086/) (2022)
- [Fix-A-Step: Semi-supervised Learning from Uncurated Unlabeled Data](https://mlsanity.com/paper/2208.11870/) (2022)
- [Label-efficient Hybrid-supervised Learning for Medical Image Segmentation](https://mlsanity.com/paper/2203.05956/) (2022)
- [RoPAWS: Robust Semi-supervised Representation Learning from Uncurated Data](https://mlsanity.com/paper/2302.14483/) (2023)
- [Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety](https://mlsanity.com/paper/2506.05451/) (2025)
- [CA-SSL: Class-Agnostic Semi-Supervised Learning for Detection and Segmentation](https://mlsanity.com/paper/2112.04966/) (2021)
- [MaxMatch: Semi-Supervised Learning with Worst-Case Consistency](https://mlsanity.com/paper/2209.12611/) (2022)
- [Total Variation Optimization Layers for Computer Vision](https://mlsanity.com/paper/2204.03643/) (2022)
- [Conformal Credal Self-Supervised Learning](https://mlsanity.com/paper/2205.15239/) (2022)
- [Learning to Predict Gradients for Semi-Supervised Continual Learning](https://mlsanity.com/paper/2201.09196/) (2022)
- [Out-of-Distributed Semantic Pruning for Robust Semi-Supervised Learning](https://mlsanity.com/paper/2305.18158/) (2023)
- [On the Importance of Calibration in Semi-supervised Learning](https://mlsanity.com/paper/2210.04783/) (2022)
- [FLiText: A Faster and Lighter Semi-Supervised Text Classification with Convolution Networks](https://mlsanity.com/paper/2110.11869/) (2021)
- [Semi-supervised Learning using Robust Loss](https://mlsanity.com/paper/2203.01524/) (2022)
- [CEIP: Combining Explicit and Implicit Priors for Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/2210.09496/) (2022)
- [Local or Global: Selective Knowledge Assimilation for Federated Learning with Limited Labels](https://mlsanity.com/paper/2307.08809/) (2023)

## Abstract (arXiv)

Existing semi-supervised learning (SSL) algorithms use a single weight to balance the loss of labeled and unlabeled examples, i.e., all unlabeled examples are equally weighted. But not all unlabeled data are equal. In this paper we study how to use a different weight for every unlabeled example. Manual tuning of all those weights -- as done in prior work -- is no longer possible. Instead, we adjust those weights via an algorithm based on the influence function, a measure of a model's dependency on one training example. To make the approach efficient, we propose a fast and effective approximation of the influence function. We demonstrate that this technique outperforms state-of-the-art methods on semi-supervised image and language classification tasks.