mlsanityData to 14 Oct 2026

Out-of-Distributed Semantic Pruning for Robust Semi-Supervised Learning

arXiv 2305.18158 ↗First posted 29 May 2023cs.CV cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Recent advances in robust semi-supervised learning (SSL) typically filter out-of-distribution (OOD) information at the sample level. We argue that an overlooked problem of robust SSL is its corrupted information on semantic level, practically limiting the development of the field. In this paper, we take an initial step to explore and propose a unified framework termed OOD Semantic Pruning (OSP), which aims at pruning OOD semantics out from in-distribution (ID) features. Specifically, (i) we propose an aliasing OOD matching module to pair each ID sample with an OOD sample with semantic overlap. (ii) We design a soft orthogonality regularization, which first transforms each ID feature by suppressing its semantic component that is collinear with paired OOD sample. It then forces the predictions before and after soft orthogonality decomposition to be consistent. Being practically simple, our method shows a strong performance in OOD detection and ID classification on challenging benchmarks. In particular, OSP surpasses the previous state-of-the-art by 13.7% on accuracy for ID classification and 5.9% on AUROC for OOD detection on TinyImageNet dataset. The source codes are publicly available at https://github.com/rain305f/OSP.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 46 of its 70 references that are in the map. First posted on arXiv 29 May 2023.

Builds on

46 of 70 references.

References · 46

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  4. 2016Wide Residual Networkscited by 3,070
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  6. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  7. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  8. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  9. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
  10. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  11. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  12. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
34 more
  1. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  2. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,057
  3. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 920
  4. 2017Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning resultscited by 472
  5. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  6. 2020Meta Pseudo Labelscited by 374
  7. 2019ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoringcited by 349
  8. 2021In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learningcited by 301
  9. 2019Label Propagation for Deep Semi-supervised Learningcited by 289
  10. 2019Interpolation Consistency Training for Semi-Supervised Learningcited by 251
  11. 2021Open-World Semi-Supervised Learningcited by 165
  12. 2018Deep Co-Training for Semi-Supervised Image Recognitioncited by 147
  13. 2020Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learningcited by 105
  14. 2020Multi-Task Curriculum Framework for Open-Set Semi-Supervised Learningcited by 82
  15. 2022Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representationscited by 57
  16. 2020Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learningcited by 53
  17. 2022Semi-Supervised and Unsupervised Deep Visual Learning: A Surveycited by 52
  18. 2022Semi-supervised Vision Transformers at Scalecited by 49
  19. 2023Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learningcited by 45
  20. 2021OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outlierscited by 43
  21. 2022ACSeg: Adaptive Conceptualization for Unsupervised Semantic Segmentationcited by 42
  22. 2021Trash to Treasure: Harvesting OOD Data with Cross-Modal Matching for Open-Set Semi-Supervised Learningcited by 40
  23. 2023Text-Video Retrieval with Disentangled Conceptualization and Set-to-Set Alignmentcited by 32
  24. 2020A Three-Stage Self-Training Framework for Semi-Supervised Semantic Segmentationcited by 30
  25. 2023Parallel Vertex Diffusion for Unified Visual Groundingcited by 28
  26. 2022Out-of-Candidate Rectification for Weakly Supervised Semantic Segmentationcited by 26
  27. 2022M3ST: Mix at Three Levels for Speech Translationcited by 21
  28. 2023PoseRAC: Pose Saliency Transformer for Repetitive Action Countingcited by 17
  29. 2020Towards Accurate and Robust Domain Adaptation under Noisy Environmentscited by 12
  30. 2023SSVMR: Saliency-based Self-training for Video-Music Retrievalcited by 12
  31. 2020Unifying Neural Learning and Symbolic Reasoning for Spinal Medical Report Generationcited by 10
  32. 2020Out-distribution aware Self-training in an Open World Settingcited by 6
  33. 2021An Empirical Study and Analysis on Open-Set Semi-Supervised Learningcited by 6
  34. 2022Robust Deep Semi-Supervised Learning: A Brief Introductioncited by 3

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 5.

7 papers.

  1. 2025MOS: Modeling Object-Scene Associations in Generalized Category Discoverycited by 10
  2. 2025Towards Effective and Efficient Context-aware Nucleus Detection in Histopathology Whole Slide Imagescited by 5
  3. 2024ProSub: Probabilistic Open-Set Semi-Supervised Learning with Subspace-Based Out-of-Distribution Detectioncited by 3
  4. 2025Re-Evaluating the Impact of Unseen-Class Unlabeled Data on Semi-Supervised Learning Modelcited by 3
  5. 2025Unsupervised Learning for Class Distribution Mismatchcited by 2
  6. 2025Diversify and Conquer: Open-set Disagreement for Robust Semi-supervised Learning with Outlierscited by 1
  7. 2024Diverse Teacher-Students for Deep Safe Semi-Supervised Learning under Class Mismatchcited by 0