# Robust Semi-supervised Learning by Wisely Leveraging Open-set Data

arXiv: https://arxiv.org/abs/2405.06979 · HTML: https://mlsanity.com/paper/2405.06979/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 75 references that are in the map.
- First posted on arXiv 11 May 2024.

## Builds on (most cited first, 30 of 43)

- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [A Convergence Theory for Deep Learning via Over-Parameterization](https://mlsanity.com/paper/1811.03962/) (2018)
- [Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming](https://mlsanity.com/paper/1309.5549/) (2013)
- [Out-of-Distribution Detection with Deep Nearest Neighbors](https://mlsanity.com/paper/2204.06507/) (2022)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Condition](https://mlsanity.com/paper/1608.04636/) (2016)
- [CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances](https://mlsanity.com/paper/2007.08176/) (2020)
- [ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring](https://mlsanity.com/paper/1911.09785/) (2019)
- [VOS: Learning What You Don't Know by Virtual Outlier Synthesis](https://mlsanity.com/paper/2202.01197/) (2022)
- [OpenOOD: Benchmarking Generalized Out-of-Distribution Detection](https://mlsanity.com/paper/2210.07242/) (2022)
- [Mitigating Neural Network Overconfidence with Logit Normalization](https://mlsanity.com/paper/2205.09310/) (2022)
- [FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning](https://mlsanity.com/paper/2205.07246/) (2022)
- [Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2102.03526/) (2021)
- [Dash: Semi-Supervised Learning with Dynamic Thresholding](https://mlsanity.com/paper/2109.00650/) (2021)
- [Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancy](https://mlsanity.com/paper/1908.04951/) (2019)
- [Semantically Coherent Out-of-Distribution Detection](https://mlsanity.com/paper/2108.11941/) (2021)
- [Stability and Generalization of Learning Algorithms that Converge to Global Optima](https://mlsanity.com/paper/1710.08402/) (2017)
- [Multi-Task Curriculum Framework for Open-Set Semi-Supervised Learning](https://mlsanity.com/paper/2007.11330/) (2020)
- [OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outliers](https://mlsanity.com/paper/2105.14148/) (2021)

## Built on by (most cited first, 20 of 20)

- [A Square Peg in a Square Hole: Meta-Expert for Long-Tailed Semi-Supervised Learning](https://mlsanity.com/paper/2505.16341/) (2025)
- [Proposal Report for the 2nd SciCAP Competition 2024](https://mlsanity.com/paper/2407.01897/) (2024)
- [Multi-level Reliable Guidance for Unpaired Multi-view Clustering](https://mlsanity.com/paper/2407.01247/) (2024)
- [ITP: Instance-Aware Test Pruning for Out-of-Distribution Detection](https://mlsanity.com/paper/2412.12566/) (2024)
- [Open set label noise learning with robust sample selection and margin-guided module](https://mlsanity.com/paper/2501.04269/) (2025)
- [The Championship-Winning Solution for the 5th CLVISION Challenge 2024](https://mlsanity.com/paper/2406.16615/) (2024)
- [First Place Solution of 2023 Global Artificial Intelligence Technology Innovation Competition Track 1](https://mlsanity.com/paper/2407.01271/) (2024)
- [Hybrid Feature Collaborative Reconstruction Network for Few-Shot Fine-Grained Image Classification](https://mlsanity.com/paper/2407.02123/) (2024)
- [Adaptive Step-size Perception Unfolding Network with Non-local Hybrid Attention for Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2407.04024/) (2024)
- [E2MPL:An Enduring and Efficient Meta Prompt Learning Framework for Few-shot Unsupervised Domain Adaptation](https://mlsanity.com/paper/2407.04066/) (2024)
- [FIPGNet:Pyramid grafting network with feature interaction strategies](https://mlsanity.com/paper/2407.04085/) (2024)
- [Second Place Solution of WSDM2023 Toloka Visual Question Answering Challenge](https://mlsanity.com/paper/2407.04255/) (2024)
- [The Solution for the AIGC Inference Performance Optimization Competition](https://mlsanity.com/paper/2407.04991/) (2024)
- [The Solution for the sequential task continual learning track of the 2nd Greater Bay Area International Algorithm Competition](https://mlsanity.com/paper/2407.04996/) (2024)
- [Imbalanced Semi-Supervised Learning via Label Refinement and Threshold Adjustment](https://mlsanity.com/paper/2407.05370/) (2024)
- [Solution for OOD-CV Workshop SSB Challenge 2024 (Open-Set Recognition Track)](https://mlsanity.com/paper/2409.20277/) (2024)
- [Toward Robust Semi-supervised Regression via Dual-stream Knowledge Distillation](https://mlsanity.com/paper/2508.14082/) (2025)
- [Dimensionally Reduced Open-World Clustering: DROWCULA](https://mlsanity.com/paper/2509.07184/) (2025)
- [Variational Rectification Inference for Learning with Noisy Labels](https://mlsanity.com/paper/2603.17255/) (2026)
- [Geometric Gradient Rectification for Safe Open-Set Semi-Supervised Learning](https://mlsanity.com/paper/2606.26973/) (2026)

## Abstract (arXiv)

Open-set Semi-supervised Learning (OSSL) holds a realistic setting that unlabeled data may come from classes unseen in the labeled set, i.e., out-of-distribution (OOD) data, which could cause performance degradation in conventional SSL models. To handle this issue, except for the traditional in-distribution (ID) classifier, some existing OSSL approaches employ an extra OOD detection module to avoid the potential negative impact of the OOD data. Nevertheless, these approaches typically employ the entire set of open-set data during their training process, which may contain data unfriendly to the OSSL task that can negatively influence the model performance. This inspires us to develop a robust open-set data selection strategy for OSSL. Through a theoretical understanding from the perspective of learning theory, we propose Wise Open-set Semi-supervised Learning (WiseOpen), a generic OSSL framework that selectively leverages the open-set data for training the model. By applying a gradient-variance-based selection mechanism, WiseOpen exploits a friendly subset instead of the whole open-set dataset to enhance the model's capability of ID classification. Moreover, to reduce the computational expense, we also propose two practical variants of WiseOpen by adopting low-frequency update and loss-based selection respectively. Extensive experiments demonstrate the effectiveness of WiseOpen in comparison with the state-of-the-art.