mlsanityData to 7 Oct 2026

Self-Training: A Survey

arXiv 2202.12040 ↗First posted 24 Feb 2022cs.LGcited by 85

Not on the map. Why?

Abstract (arXiv)

Semi-supervised algorithms aim to learn prediction functions from a small set of labeled observations and a large set of unlabeled observations. Because this framework is relevant in many applications, they have received a lot of interest in both academia and industry. Among the existing techniques, self-training methods have undoubtedly attracted greater attention in recent years. These models are designed to find the decision boundary on low density regions without making additional assumptions about the data distribution, and use the unsigned output score of a learned classifier, or its margin, as an indicator of confidence. The working principle of self-training algorithms is to learn a classifier iteratively by assigning pseudo-labels to the set of unlabeled training samples with a margin greater than a certain threshold. The pseudo-labeled examples are then used to enrich the labeled training data and to train a new classifier in conjunction with the labeled training set. In this paper, we present self-training methods for binary and multi-class classification; as well as their variants and two related approaches, namely consistency-based approaches and transductive learning. We examine the impact of significant self-training features on various methods, using different general and image classification benchmarks, and we discuss our ideas for future research in self-training. To the best of our knowledge, this is the first thorough and complete survey on this subject.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Builds on 43 of its 136 references that are in the map. First posted on arXiv 24 Feb 2022.

Builds on

43 of 136 references.

References · 43

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2016Wide Residual Networkscited by 3,057
  3. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  4. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  5. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  6. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  7. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  8. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  9. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  10. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
  11. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 537
  12. 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
31 more
  1. 2021Semi-Supervised Semantic Segmentation with Cross Pseudo Supervisioncited by 462
  2. 2019Confidence Regularized Self-Trainingcited by 418
  3. 2021A Survey on Deep Semi-supervised Learningcited by 250
  4. 2018Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptationcited by 218
  5. 2020ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learningcited by 209
  6. 2020Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Datacited by 179
  7. 2017Asymmetric Tri-training for Unsupervised Domain Adaptationcited by 152
  8. 2020Predicting What You Already Know Helps: Provable Self-Supervised Learningcited by 152
  9. 2020Understanding Self-Training for Gradual Domain Adaptationcited by 149
  10. 2021MIST: Multiple Instance Self-Training Framework for Video Anomaly Detectioncited by 137
  11. 2019Self-Training for End-to-End Speech Recognitioncited by 128
  12. 2020Enabling Language Models to Fill in the Blankscited by 121
  13. 2021Cycle Self-Training for Domain Adaptationcited by 120
  14. 2020Self-training and Pre-training are Complementary for Speech Recognitioncited by 98
  15. 2017Recovery Guarantees for One-hidden-layer Neural Networkscited by 94
  16. 2021Multi-Task Self-Training for Learning General Representationscited by 85
  17. 2020Self-training Avoids Using Spurious Features Under Domain Shiftcited by 69
  18. 2022Corrupted Image Modeling for Self-Supervised Visual Pre-Trainingcited by 68
  19. 2022Debiased Self-Training for Semi-Supervised Learningcited by 62
  20. 2021Self-Training with Weak Supervisioncited by 49
  21. 2020Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Languagecited by 37
  22. 2021Self-training Converts Weak Learners to Strong Learners in Mixture Modelscited by 19
  23. 2022How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysiscited by 18
  24. 2016Rademacher Complexity Bounds for a Penalized Multiclass Semi-Supervised Algorithmcited by 15
  25. 2021Transductive Robust Learning Guaranteescited by 10
  26. 2021Unsupervised Self-Training for Sentiment Analysis of Code-Switched Datacited by 7
  27. 2017Aggressive Sampling for Multi-class to Binary Reduction with Applications to Text Classificationcited by 3
  28. 2021Machine Learning Applications for Therapeutic Tasks with Genomics Datacited by 3
  29. 2021Multi-class Probabilistic Bounds for Self-learningcited by 3
  30. 2021Improving Pseudo-label Training For End-to-end Speech Recognition Using Gradient Maskcited by 3
  31. 2021Self-Training of Halfspaces with Generalization Guarantees under Massart Mislabeling Noise Modelcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 20; 2024: 20; 2025: 26; 2026: 11.

85 papers.

Sort:

By area:

  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  2. 2024Toward Visual Grounding: A Surveycited by 52
  3. 2025VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Datacited by 39
  4. 2026Embarrassingly Simple Self-Distillation Improves Code Generationcited by 36
  5. 2024West-of-N: Synthetic Preferences for Self-Improving Reward Modelscited by 27
  6. 2023Balancing Logit Variation for Long-tailed Semantic Segmentationcited by 26
  7. 2024Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoningcited by 26
  8. 2025Reasoning-Enhanced Self-Training for Long-Form Personalized Text Generationcited by 26
  9. 2024STEP: Enhancing Video-LLMs' Compositional Reasoning by Spatio-Temporal Graph-guided Self-Trainingcited by 22
  10. 2024A Review of Pseudo-Labeling for Computer Visioncited by 16
  11. 2024Rethinking Chain-of-Thought from the Perspective of Self-Trainingcited by 10
  12. 2026From Memorization to Creativity: LLM as a Designer of Novel Neural Architecturescited by 10
38 more
  1. 2024MANO: Exploiting Matrix Norm for Unsupervised Accuracy Estimation Under Distribution Shiftscited by 8
  2. 2025OntoTune: Ontology-Driven Self-training for Aligning Large Language Modelscited by 8
  3. 2022Fraud Dataset Benchmark and Applicationscited by 7
  4. 2023Leveraging Ensemble Diversity for Robust Self-Training in the Presence of Sample Selection Biascited by 7
  5. 2024LG AI Research & KAIST at EHRSQL 2024: Self-Training Large Language Models with Pseudo-Labeled Unanswerable Questions for a Reliable Text-to-SQL System on EHRscited by 7
  6. 2023Out-of-Domain Intent Detection Considering Multi-Turn Dialogue Contextscited by 6
  7. 2025Dual Classification Head Self-training Network for Cross-scene Hyperspectral Image Classificationcited by 6
  8. 2025Revisiting semi-supervised learning in the era of foundation modelscited by 6
  9. 2023Unsupervised Prototype Adapter for Vision-Language Modelscited by 5
  10. 2022Voxels Intersecting along Orthogonal Levels Attention U-Net for Intracerebral Haemorrhage Segmentation in Head CTcited by 4
  11. 2024Retraining with Predicted Hard Labels Provably Increases Model Accuracycited by 4
  12. 2024Semi-Supervised Object Detection: A Survey on Progress from CNN to Transformercited by 4
  13. 2025ST-SAM: SAM-Driven Self-Training Framework for Semi-Supervised Camouflaged Object Detectioncited by 4
  14. 2023Uncertainty-aware Self-training for Low-resource Neural Sequence Labelingcited by 3
  15. 2023Domain-Adaptive Full-Face Gaze Estimation via Novel-View-Synthesis and Feature Disentanglementcited by 3
  16. 2023Uncertainty-aware Parameter-Efficient Self-training for Semi-supervised Language Understandingcited by 3
  17. 2023A Review of Machine Learning Methods Applied to Video Analysis Systemscited by 3
  18. 2024Leveraging Gradients for Unsupervised Accuracy Estimation under Distribution Shiftcited by 3
  19. 2024Time-Reversal Provides Unsupervised Feedback to LLMscited by 3
  20. 2024PyPotteryLens: An Open-Source Deep Learning Framework for Automated Digitisation of Archaeological Pottery Documentationcited by 3
  21. 2025Semi-rPPG: Semi-Supervised Remote Physiological Measurement with Curriculum Pseudo-Labelingcited by 3
  22. 2022Tunable Hybrid Proposal Networks for the Open Worldcited by 2
  23. 2022Dependency-aware Self-training for Entity Alignmentcited by 2
  24. 2023Data Augmentation for Neural NLPcited by 2
  25. 2023Lifelong Change Detection: Continuous Domain Adaptation for Small Object Change Detection in Every Robot Navigationcited by 2
  26. 2025Context-Aware Doubly-Robust Semi-Supervised Learningcited by 2
  27. 2025Chart-CoCa: Self-Improving Chart Understanding of Vision LMs via Code-Driven Synthesis and Candidate-Conditioned Answeringcited by 2
  28. 2026Triple Expert Learning from Noisy Labels for Semi-Supervised Vision Foundation Model Adaptationcited by 2
  29. 2022Large-Scale Sequential Learning for Recommender and Engineering Systemscited by 1
  30. 2023How to choose "Good" Samples for Text Data Augmentationcited by 1
  31. 2023HugNLP: A Unified and Comprehensive Library for Natural Language Processingcited by 1
  32. 2023Augment and Criticize: Exploring Informative Samples for Semi-Supervised Monocular 3D Object Detectioncited by 1
  33. 2023Intent-calibrated Self-training for Answer Selection in Open-domain Dialoguescited by 1
  34. 2024Probing Perfection: The Relentless Art of Meddling for Pulmonary Airway Segmentation from HRCT via a Human-AI Collaboration Based Active Learning Methodcited by 1
  35. 2024When does a bridge become an aeroplane?cited by 1
  36. 2025REFINE-AF: A Task-Agnostic Framework to Align Language Models via Self-Generated Instructions using Reinforcement Learning from Automated Feedbackcited by 1
  37. 2025ADAPT: A Pseudo-labeling Approach to Combat Concept Drift in Malware Detectioncited by 1
  38. 2025Training-Free Label Space Alignment for Universal Domain Adaptationcited by 1

50 most cited of 85 All 85 (JSON)