mlsanityData to 7 Oct 2026

Unsupervised Data Augmentation for Consistency Training

arXiv 1904.12848 ↗First posted 29 Apr 2019cs.LG cs.AI cs.CLcited by 1,057

Not on the map. Why?

Abstract (arXiv)

Semi-supervised learning lately has shown much promise in improving deep learning models when labeled data is scarce. Common among recent approaches is the use of consistency training on a large amount of unlabeled data to constrain model predictions to be invariant to input noise. In this work, we present a new perspective on how to effectively noise unlabeled examples and argue that the quality of noising, specifically those produced by advanced data augmentation methods, plays a crucial role in semi-supervised learning. By substituting simple noising operations with advanced data augmentation methods such as RandAugment and back-translation, our method brings substantial improvements across six language and three vision tasks under the same consistency training framework. On the IMDb text classification dataset, with only 20 labeled examples, our method achieves an error rate of 4.20, outperforming the state-of-the-art model trained on 25,000 labeled examples. On a standard semi-supervised learning benchmark, CIFAR-10, our method outperforms all previous approaches and achieves an error rate of 5.43 with only 250 examples. Our method also combines well with transfer learning, e.g., when finetuning from BERT, and yields improvements in high-data regime, such as ImageNet, whether when there is only 10% labeled data or when a full labeled set with 1.3M extra unlabeled examples is used. Code is available at https://github.com/google-research/uda.

Key facts. Cited by 1,057 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 260; 2022: 249; 2023: 243; 2024: 161; 2025: 96; 2026: 47.

1,057 papers.

Sort:

By area:

  1. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  2. 2024Depth Anything V2cited by 1,385
  3. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  4. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  5. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 537
  6. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  7. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  8. 2021True Few-Shot Learning with Language Modelscited by 343
  9. 2021End-to-End Semi-Supervised Object Detection with Soft Teachercited by 315
  10. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 311
  11. 2021A Survey on Deep Semi-supervised Learningcited by 250
  12. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
38 more
  1. 2022FreeMatch: Self-adaptive Thresholding for Semi-supervised Learningcited by 232
  2. 2021TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentationcited by 225
  3. 2021One Million Scenes for Autonomous Driving: ONCE Datasetcited by 218
  4. 2022Is GPT-3 a Good Data Annotator?cited by 208
  5. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  6. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 185
  7. 2021ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentationcited by 184
  8. 2021Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classificationcited by 183
  9. 2022Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentationcited by 177
  10. 2021CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learningcited by 174
  11. 2021Noisy Channel Language Model Prompting for Few-Shot Text Classificationcited by 163
  12. 2021Self-supervised Augmentation Consistency for Adapting Semantic Segmentationcited by 149
  13. 2021Dash: Semi-Supervised Learning with Dynamic Thresholdingcited by 147
  14. 2021Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalizationcited by 141
  15. 2023SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learningcited by 133
  16. 2022Data Augmentation for Deep Graph Learning: A Surveycited by 132
  17. 2021Entailment as Few-Shot Learnercited by 122
  18. 2022SimMatch: Semi-supervised Learning with Similarity Matchingcited by 118
  19. 2021Humble Teachers Teach Better Students for Semi-Supervised Object Detectioncited by 115
  20. 2021Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learnerscited by 115
  21. 2021Instant-Teaching: An End-to-End Semi-Supervised Object Detection Frameworkcited by 112
  22. 2021Understanding and Improving Early Stopping for Learning with Noisy Labelscited by 112
  23. 2021A Survey on Data Augmentation for Text Classificationcited by 111
  24. 2021Mutual Consistency Learning for Semi-supervised Medical Image Segmentationcited by 107
  25. 2021Semi-supervised Left Atrium Segmentation with Mutual Consistency Trainingcited by 104
  26. 2021Recent advances and clinical applications of deep learning in medical image analysiscited by 104
  27. 2021AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptationcited by 104
  28. 2023Text Classification via Large Language Modelscited by 104
  29. 2021Self-Guided Contrastive Learning for BERT Sentence Representationscited by 103
  30. 2021An Empirical Survey of Data Augmentation for Limited Data Learning in NLPcited by 101
  31. 2021Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentationcited by 101
  32. 2021Semi-Supervised Semantic Segmentation via Adaptive Equalization Learningcited by 101
  33. 2021Improving and Simplifying Pattern Exploiting Trainingcited by 97
  34. 2022SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learningcited by 96
  35. 2022A Comprehensive Survey of Image Augmentation Techniques for Deep Learningcited by 95
  36. 2022USB: A Unified Semi-supervised Learning Benchmark for Classificationcited by 95
  37. 2022Robust Training under Label Noise by Over-parameterizationcited by 93
  38. 2021Towards Principled Disentanglement for Domain Generalizationcited by 91

50 most cited of 1,057 All 1,057 (JSON)