# Trash to Treasure: Harvesting OOD Data with Cross-Modal Matching for Open-Set Semi-Supervised Learning

arXiv: https://arxiv.org/abs/2108.05617 · HTML: https://mlsanity.com/paper/2108.05617/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 47 references that are in the map.
- First posted on arXiv 12 Aug 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [S4L: Self-Supervised Semi-Supervised Learning](https://mlsanity.com/paper/1905.03670/) (2019)
- [Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks](https://mlsanity.com/paper/1406.6909/) (2014)
- [Label Propagation for Deep Semi-supervised Learning](https://mlsanity.com/paper/1904.04717/) (2019)
- [Revisiting Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/1901.09005/) (2019)
- [Deep Co-Training for Semi-Supervised Image Recognition](https://mlsanity.com/paper/1803.05984/) (2018)
- [Multi-Task Curriculum Framework for Open-Set Semi-Supervised Learning](https://mlsanity.com/paper/2007.11330/) (2020)

## Built on by (most cited first, 30 of 40)

- [Learning Semi-supervised Gaussian Mixture Models for Generalized Category Discovery](https://mlsanity.com/paper/2305.06144/) (2023)
- [Semi-Supervised and Unsupervised Deep Visual Learning: A Survey](https://mlsanity.com/paper/2208.11296/) (2022)
- [Harnessing Out-Of-Distribution Examples via Augmenting Content and Style](https://mlsanity.com/paper/2207.03162/) (2022)
- [OpenCon: Open-world Contrastive Learning](https://mlsanity.com/paper/2208.02764/) (2022)
- [Incremental Generalized Category Discovery](https://mlsanity.com/paper/2304.14310/) (2023)
- [IOMatch: Simplifying Open-Set Semi-Supervised Learning with Joint Inliers and Outliers Utilization](https://mlsanity.com/paper/2308.13168/) (2023)
- [A Graph-Theoretic Framework for Understanding Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2311.03524/) (2023)
- [Robust Semi-supervised Learning by Wisely Leveraging Open-set Data](https://mlsanity.com/paper/2405.06979/) (2024)
- [Open-Set Semi-Supervised Object Detection](https://mlsanity.com/paper/2208.13722/) (2022)
- [Open-world Semi-supervised Novel Class Discovery](https://mlsanity.com/paper/2305.13095/) (2023)
- [Effective Prompt Pool Learning for Continual Category Discovery](https://mlsanity.com/paper/2407.19001/) (2024)
- [Fix-A-Step: Semi-supervised Learning from Uncurated Unlabeled Data](https://mlsanity.com/paper/2208.11870/) (2022)
- [RoPAWS: Robust Semi-supervised Representation Learning from Uncurated Data](https://mlsanity.com/paper/2302.14483/) (2023)
- [Knowledge Distillation Meets Open-Set Semi-Supervised Learning](https://mlsanity.com/paper/2205.06701/) (2022)
- [SSB: Simple but Strong Baseline for Boosting Performance of Open-Set Semi-Supervised Learning](https://mlsanity.com/paper/2311.10572/) (2023)
- [Semi-supervised Deep Learning for Image Classification with Distribution Mismatch: A Survey](https://mlsanity.com/paper/2203.00190/) (2022)
- [Semi-Supervised Learning via Weight-aware Distillation under Class Distribution Mismatch](https://mlsanity.com/paper/2308.11874/) (2023)
- [Robust Semi-Supervised Learning in Open Environments](https://mlsanity.com/paper/2412.18256/) (2024)
- [Towards Unbiased Training in Federated Open-world Semi-supervised Learning](https://mlsanity.com/paper/2305.00771/) (2023)
- [Out-of-Distributed Semantic Pruning for Robust Semi-Supervised Learning](https://mlsanity.com/paper/2305.18158/) (2023)
- [Embedding contrastive unsupervised features to cluster in- and out-of-distribution noise in corrupted image datasets](https://mlsanity.com/paper/2207.01573/) (2022)
- [Improving Open-Set Semi-Supervised Learning with Self-Supervision](https://mlsanity.com/paper/2301.10127/) (2023)
- [On Pseudo-Labeling for Class-Mismatch Semi-Supervised Learning](https://mlsanity.com/paper/2301.06010/) (2023)
- [Robust Deep Semi-Supervised Learning: A Brief Introduction](https://mlsanity.com/paper/2202.05975/) (2022)
- [Adaptive Negative Evidential Deep Learning for Open-set Semi-supervised Learning](https://mlsanity.com/paper/2303.12091/) (2023)
- [ProSub: Probabilistic Open-Set Semi-Supervised Learning with Subspace-Based Out-of-Distribution Detection](https://mlsanity.com/paper/2407.11735/) (2024)
- [SCOMatch: Alleviating Overtrusting in Open-set Semi-supervised Learning](https://mlsanity.com/paper/2409.17512/) (2024)
- [OwMatch: Conditional Self-Labeling with Consistency for Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2411.01833/) (2024)
- [Exploration and Exploitation of Unlabeled Data for Open-Set Semi-Supervised Learning](https://mlsanity.com/paper/2306.17699/) (2023)
- [Unsupervised Learning for Class Distribution Mismatch](https://mlsanity.com/paper/2505.06948/) (2025)

## Abstract (arXiv)

Open-set semi-supervised learning (open-set SSL) investigates a challenging but practical scenario where out-of-distribution (OOD) samples are contained in the unlabeled data. While the mainstream technique seeks to completely filter out the OOD samples for semi-supervised learning (SSL), we propose a novel training mechanism that could effectively exploit the presence of OOD data for enhanced feature learning while avoiding its adverse impact on the SSL. We achieve this goal by first introducing a warm-up training that leverages all the unlabeled data, including both the in-distribution (ID) and OOD samples. Specifically, we perform a pretext task that enforces our feature extractor to obtain a high-level semantic understanding of the training images, leading to more discriminative features that can benefit the downstream tasks. Since the OOD samples are inevitably detrimental to SSL, we propose a novel cross-modal matching strategy to detect OOD samples. Instead of directly applying binary classification, we train the network to predict whether the data sample is matched to an assigned one-hot class label. The appeal of the proposed cross-modal matching over binary classification is the ability to generate a compatible feature space that aligns with the core classification task. Extensive experiments show that our approach substantially lifts the performance on open-set SSL and outperforms the state-of-the-art by a large margin.