mlsanityData to 14 Oct 2026

Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning

arXiv 1704.03976 ↗First posted 13 Apr 2017stat.ML cs.LGcited by 920

Not on the map. Why?

Cited by 29 in Knowledge distillation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose a new regularization method based on virtual adversarial loss: a new measure of local smoothness of the conditional label distribution given input. Virtual adversarial loss is defined as the robustness of the conditional label distribution around each input data point against local perturbation. Unlike adversarial training, our method defines the adversarial direction without label information and is hence applicable to semi-supervised learning. Because the directions in which we smooth the model are only "virtually" adversarial, we call our method virtual adversarial training (VAT). The computational cost of VAT is relatively low. For neural networks, the approximated gradient of virtual adversarial loss can be computed with no more than two pairs of forward- and back-propagations. In our experiments, we applied VAT to supervised and semi-supervised learning tasks on multiple benchmark datasets. With a simple enhancement of the algorithm based on the entropy minimization principle, our VAT achieves state-of-the-art performance for semi-supervised learning tasks on SVHN and CIFAR-10.

Key facts. Cited by 920 papers in the map (data through 14 Oct 2026). First posted on arXiv 13 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 270; 2022: 237; 2023: 186; 2024: 123; 2025: 66; 2026: 38.

920 papers.

Sort:

By area:

  1. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 540
  2. 2021Semi-Supervised Semantic Segmentation with Cross Pseudo Supervisioncited by 463
  3. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 434
  4. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 359
  5. 2021End-to-End Semi-Supervised Object Detection with Soft Teachercited by 316
  6. 2021Unbiased Teacher for Semi-Supervised Object Detectioncited by 306
  7. 2021In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learningcited by 301
  8. 2021A Fourier-based Framework for Domain Generalizationcited by 290
  9. 2021A Survey on Deep Semi-supervised Learningcited by 251
  10. 2023A Cookbook of Self-Supervised Learningcited by 250
  11. 2022FreeMatch: Self-adaptive Thresholding for Semi-supervised Learningcited by 232
  12. 2021One Million Scenes for Autonomous Driving: ONCE Datasetcited by 218
38 more
  1. 2021General Facial Representation Learning in a Visual-Linguistic Mannercited by 195
  2. 2021ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentationcited by 184
  3. 2022Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentationcited by 178
  4. 2021CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learningcited by 175
  5. 2021Dash: Semi-Supervised Learning with Dynamic Thresholdingcited by 148
  6. 2021Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalizationcited by 141
  7. 2023SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learningcited by 133
  8. 2021Co-Mixup: Saliency Guided Joint Mixup with Supermodular Diversitycited by 128
  9. 2021Advances in adversarial attacks and defenses in computer vision: A surveycited by 127
  10. 2021Cycle Self-Training for Domain Adaptationcited by 120
  11. 2021Causality-inspired Single-source Domain Generalization for Medical Image Segmentationcited by 119
  12. 2022SimMatch: Semi-supervised Learning with Similarity Matchingcited by 118
  13. 2021Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samplescited by 114
  14. 2021Instant-Teaching: An End-to-End Semi-Supervised Object Detection Frameworkcited by 113
  15. 2021Perturbed and Strict Mean Teachers for Semi-supervised Semantic Segmentationcited by 113
  16. 2021Semi-supervised Semantic Segmentation with Directional Context-aware Consistencycited by 112
  17. 2021Mutual Consistency Learning for Semi-supervised Medical Image Segmentationcited by 108
  18. 2021Self-Guided Contrastive Learning for BERT Sentence Representationscited by 103
  19. 2021GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injectioncited by 102
  20. 2021An Empirical Survey of Data Augmentation for Limited Data Learning in NLPcited by 101
  21. 2021Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentationcited by 101
  22. 2021Semi-Supervised Semantic Segmentation via Adaptive Equalization Learningcited by 101
  23. 20213D AffordanceNet: A Benchmark for Visual Object Affordance Understandingcited by 100
  24. 2022SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learningcited by 97
  25. 2022USB: A Unified Semi-supervised Learning Benchmark for Classificationcited by 95
  26. 2022Exploring Smoothness and Class-Separation for Semi-supervised Medical Image Segmentationcited by 93
  27. 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 87
  28. 2021Guided Point Contrastive Learning for Semi-supervised Point Cloud Semantic Segmentationcited by 87
  29. 2022Self-Training: A Surveycited by 86
  30. 2022Debiased Learning from Naturally Imbalanced Pseudo-Labelscited by 85
  31. 2022A Closer Look at Smoothness in Domain Adversarial Trainingcited by 85
  32. 2021Bootstrapping Semantic Segmentation with Regional Contrastcited by 80
  33. 2021PixMatch: Unsupervised Domain Adaptation via Pixelwise Consistency Trainingcited by 80
  34. 2021Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learningcited by 78
  35. 2021Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigationcited by 78
  36. 2023Federated Learning for Medical Image Analysis: A Surveycited by 78
  37. 2022Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentationcited by 76
  38. 2021Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labelscited by 74

50 most cited of 920 All 920 (JSON)