mlsanityData to 7 Oct 2026

A Survey on Deep Semi-supervised Learning

arXiv 2103.00550 ↗First posted 28 Feb 2021cs.LGcited by 250

Not on the map. Why?

Abstract (arXiv)

Deep semi-supervised learning is a fast-growing field with a range of practical applications. This paper provides a comprehensive survey on both fundamentals and recent advances in deep semi-supervised learning methods from perspectives of model design and unsupervised loss functions. We first present a taxonomy for deep semi-supervised learning that categorizes existing methods, including deep generative methods, consistency regularization methods, graph-based methods, pseudo-labeling methods, and hybrid methods. Then we provide a comprehensive review of 52 representative methods and offer a detailed comparison of these methods in terms of the type of losses, contributions, and architecture differences. In addition to the progress in the past few years, we further discuss some shortcomings of existing methods and provide some tentative heuristic solutions for solving these open problems.

Key facts. Cited by 250 papers in the map (data through 7 Oct 2026). Builds on 149 of its 278 references that are in the map. First posted on arXiv 28 Feb 2021.

Builds on

149 of 278 references.

References · 149

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  7. 2017Graph Attention Networkscited by 8,854
  8. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  9. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  10. 2017Inductive Representation Learning on Large Graphscited by 6,284
  11. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  12. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
38 more
  1. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  2. 2014Going Deeper with Convolutionscited by 4,231
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  5. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  6. 2016Improved Techniques for Training GANscited by 2,938
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  8. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  9. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  10. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  11. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  12. 2017Improved Training of Wasserstein GANscited by 2,223
  13. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  14. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  15. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  16. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  17. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  18. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
  19. 2019Simplifying Graph Convolutional Networkscited by 1,470
  20. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  21. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  22. 2016Variational Graph Auto-Encoderscited by 1,373
  23. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  24. 2017Random Erasing Data Augmentationcited by 1,235
  25. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  26. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  27. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
  28. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  29. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  30. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  31. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  32. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  33. 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 875
  34. 2015Gated Graph Sequence Neural Networkscited by 792
  35. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  36. 2017In Defense of the Triplet Loss for Person Re-Identificationcited by 784
  37. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  38. 2020Self-supervised Learning: Generative or Contrastivecited by 693

50 most cited of 149 All 149 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 42; 2023: 58; 2024: 61; 2025: 58; 2026: 23.

250 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  2. 2022USB: A Unified Semi-supervised Learning Benchmark for Classificationcited by 95
  3. 2023Dynamic Conceptional Contrastive Learning for Generalized Category Discoverycited by 90
  4. 2022Self-Training: A Surveycited by 85
  5. 2023Federated Learning for Medical Image Analysis: A Surveycited by 77
  6. 2022Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentationcited by 74
  7. 2022UCC: Uncertainty guided Cross-head Co-training for Semi-Supervised Semantic Segmentationcited by 45
  8. 2022Machine Learning in Aerodynamic Shape Optimizationcited by 39
  9. 2024SelEx: Self-Expertise in Fine-Grained Generalized Category Discoverycited by 34
  10. 2025Bridging the Vision-Brain Gap with an Uncertainty-Aware Blur Priorcited by 28
  11. 2022Beyond Just Vision: A Review on Self-Supervised Representation Learning on Multimodal and Temporal Datacited by 27
  12. 2023RCPS: Rectified Contrastive Pseudo Supervision for Semi-Supervised Medical Image Segmentationcited by 27
38 more
  1. 2022CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learningcited by 24
  2. 2023LEOD: Label-Efficient Object Detection for Event Camerascited by 24
  3. 2023IOMatch: Simplifying Open-Set Semi-Supervised Learning with Joint Inliers and Outliers Utilizationcited by 23
  4. 2023Five A+ Network: You Only Need 9K Parameters for Underwater Image Enhancementcited by 22
  5. 2022Prompt Tuning for Parameter-efficient Medical Image Segmentationcited by 21
  6. 2024PanDA: Towards Panoramic Depth Anything with Unlabeled Panoramas and Mobius Spatial Augmentationcited by 21
  7. 2023A Survey of Label-Efficient Deep Learning for 3D Point Cloudscited by 20
  8. 2021Domain-Specific Bias Filtering for Single Labeled Domain Generalizationcited by 18
  9. 2021DP-SSL: Towards Robust Semi-supervised Learning with A Few Labeled Samplescited by 18
  10. 2022An Information-theoretical Approach to Semi-supervised Learning under Covariate-shiftcited by 18
  11. 2024Label-Agnostic Forgetting: A Supervision-Free Unlearning in Deep Modelscited by 18
  12. 2024SemiCD-VL: Visual-Language Model Guidance Makes Better Semi-supervised Change Detectorcited by 18
  13. 2022End-to-End Semi-Supervised Learning for Video Action Detectioncited by 17
  14. 2023Consistency Regularization for Generalizable Source-free Domain Adaptationcited by 17
  15. 2023Late Stopping: Avoiding Confidently Learning from Mislabeled Examplescited by 17
  16. 2023XVO: Generalized Visual Odometry via Cross-Modal Self-Trainingcited by 17
  17. 2022Label-Efficient Online Continual Object Detection in Streaming Videocited by 16
  18. 2023A Survey on Semi-Supervised Semantic Segmentationcited by 16
  19. 2024A Review of Pseudo-Labeling for Computer Visioncited by 16
  20. 2024Lessons and Insights from a Unifying Study of Parameter-Efficient Fine-Tuning (PEFT) in Visual Recognitioncited by 16
  21. 2022Discovering Human-Object Interaction Concepts via Self-Compositional Learningcited by 15
  22. 2023Dynamic Graph Representation Learning with Neural Networks: A Surveycited by 15
  23. 2023Neighborhood-Regularized Self-Training for Learning with Few Labelscited by 14
  24. 2023Label-Efficient Deep Learning in Medical Image Analysis: Challenges and Future Directionscited by 14
  25. 2024Semi-Supervised Risk Control via Prediction-Powered Inferencecited by 13
  26. 2023SYNAuG: Exploiting Synthetic Data for Data Imbalance Problemscited by 12
  27. 2025STiL: Semi-supervised Tabular-Image Learning for Comprehensive Task-Relevant Information Exploration in Multimodal Classificationcited by 12
  28. 2022On Non-Random Missing Labels in Semi-Supervised Learningcited by 11
  29. 2024Exploring the Benefits of Vision Foundation Models for Unsupervised Domain Adaptationcited by 11
  30. 2025CGMatch: A Different Perspective of Semi-supervised Learningcited by 11
  31. 2023Unified Convergence Theory of Stochastic and Variance-Reduced Cubic Newton Methodscited by 10
  32. 2023Semi-supervised Active Learning for Video Action Detectioncited by 10
  33. 2024Computer Vision for Primate Behavior Analysis in the Wildcited by 10
  34. 2024Dual Mean-Teacher: An Unbiased Semi-Supervised Framework for Audio-Visual Source Localizationcited by 10
  35. 2024Rethinking Chain-of-Thought from the Perspective of Self-Trainingcited by 10
  36. 2022Instance-dependent Label Distribution Estimation for Learning with Label Noisecited by 9
  37. 2023SLaM: Student-Label Mixing for Distillation with Unlabeled Examplescited by 9
  38. 2023Model-free Test Time Adaptation for Out-Of-Distribution Detectioncited by 9

50 most cited of 250 All 250 (JSON)