mlsanityData to 14 Oct 2026

On Data-Augmentation and Consistency-Based Semi-Supervised Learning

arXiv 2101.06967 ↗First posted 18 Jan 2021stat.ML cs.LG stat.APcited by 13

Not on the map. Why?

Abstract (arXiv)

Recently proposed consistency-based Semi-Supervised Learning (SSL) methods such as the Pi-model, temporal ensembling, the mean teacher, or the virtual adversarial training, have advanced the state of the art in several SSL tasks. These methods can typically reach performances that are comparable to their fully supervised counterparts while using only a fraction of labelled examples. Despite these methodological advances, the understanding of these methods is still relatively limited. In this text, we analyse (variations of) the Pi-model in settings where analytically tractable results can be obtained. We establish links with Manifold Tangent Classifiers and demonstrate that the quality of the perturbations is key to obtaining reasonable SSL performances. Importantly, we propose a simple extension of the Hidden Manifold Model that naturally incorporates data-augmentation schemes and offers a framework for understanding and experimenting with SSL methods.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 26 of its 57 references that are in the map. First posted on arXiv 18 Jan 2021.

Builds on

26 of 57 references.

References · 26

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  4. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  5. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
  6. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  7. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  8. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,057
  9. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 920
  10. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  11. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 763
  12. 2018AutoAugment: Learning Augmentation Policies from Datacited by 730
14 more
  1. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  2. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 503
  3. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  4. 2017The Effectiveness of Data Augmentation in Image Classification using Deep Learningcited by 370
  5. 2014Learning with Pseudo-Ensemblescited by 241
  6. 2017High-dimensional dynamics of generalization error in neural networkscited by 241
  7. 2017Unsupervised Neural Machine Translationcited by 149
  8. 2019Learning Data Augmentation Strategies for Object Detectioncited by 139
  9. 2020Generalisation error in learning with random features and the hidden manifold modelcited by 127
  10. 2017Smart Augmentation - Learning an Optimal Data Augmentation Strategycited by 54
  11. 2017Smooth Neighbors on Teacher Graphs for Semi-supervised Learningcited by 48
  12. 2016Deep Learning and Hierarchal Generative Modelscited by 25
  13. 2018Large Data and Zero Noise Limits of Graph-Based Semi-Supervised Learning Algorithmscited by 22
  14. 2019Modelling the influence of data structure on learning in neural networks: the hidden manifold modelcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 5; 2023: 3; 2024: 0; 2025: 0; 2026: 1.

13 papers.

Sort:

  1. 2021A Survey on Deep Semi-supervised Learningcited by 251
  2. 2022Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentationcited by 76
  3. 2021OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outlierscited by 43
  4. 2021Multimodal Semi-Supervised Learning for 3D Objectscited by 24
  5. 2022Contrastive Regularization for Semi-Supervised Learningcited by 24
  6. 2023From Semi-supervised to Omni-supervised Room Layout Estimation Using Point Cloudscited by 18
  7. 2022Class-Level Confidence Based 3D Semi-Supervised Learningcited by 14
  8. 2022PARSE: Pairwise Alignment of Representations in Semi-Supervised EEG Learning for Emotion Recognitioncited by 12
  9. 2022Toward Learning Robust and Invariant Representations with Alignment Regularization and Data Augmentationcited by 11
  10. 2023SCoDA: Domain Adaptive Shape Completion for Real Scanscited by 9
  11. 2021An Empirical Study and Analysis on Open-Set Semi-Supervised Learningcited by 6
  12. 2023Semi-supervised Relation Extraction via Data Augmentation and Consistency-trainingcited by 1
1 more
  1. 2026MTSSL: Meta-Thresholding Semi-Supervised Learningcited by 0