mlsanityData to 14 Oct 2026

Revisiting Consistency Regularization for Semi-Supervised Learning

arXiv 2112.05825 ↗First posted 10 Dec 2021cs.CVcited by 34

Not on the map. Why?

Abstract (arXiv)

Consistency regularization is one of the most widely-used techniques for semi-supervised learning (SSL). Generally, the aim is to train a model that is invariant to various data augmentations. In this paper, we revisit this idea and find that enforcing invariance by decreasing distances between features from differently augmented images leads to improved performance. However, encouraging equivariance instead, by increasing the feature distance, further improves performance. To this end, we propose an improved consistency regularization framework by a simple yet effective technique, FeatDistLoss, that imposes consistency and equivariance on the classifier and the feature level, respectively. Experimental results show that our model defines a new state of the art for various datasets and settings and outperforms previous work by a significant margin, particularly in low data regimes. Extensive experiments are conducted to analyze the method, and the code will be published.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Builds on 54 of its 78 references that are in the map. First posted on arXiv 10 Dec 2021.

Builds on

54 of 78 references.

References · 54

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  9. 2016Wide Residual Networkscited by 3,070
  10. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  11. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  12. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
38 more
  1. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  2. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
  3. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  4. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
  5. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  6. 2016Group Equivariant Convolutional Networkscited by 1,165
  7. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  8. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,057
  9. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  10. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 989
  11. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 920
  12. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 809
  13. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  14. 2017Dynamic Routing Between Capsulescited by 689
  15. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 540
  16. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 510
  17. 2014Semi-Supervised Learning with Deep Generative Modelscited by 496
  18. 2017Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning resultscited by 472
  19. 2019General E(2)-Equivariant Steerable CNNscited by 446
  20. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  21. 2020Meta Pseudo Labelscited by 374
  22. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  23. 2016Steerable CNNscited by 305
  24. 2019Label Propagation for Deep Semi-supervised Learningcited by 289
  25. 2019Interpolation Consistency Training for Semi-Supervised Learningcited by 251
  26. 2015Semi-Supervised Learning with Ladder Networkscited by 247
  27. 2014Learning with Pseudo-Ensemblescited by 241
  28. 2021CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learningcited by 175
  29. 2020Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learningcited by 105
  30. 2016Semi-Supervised Learning with Generative Adversarial Networkscited by 103
  31. 2021ABC: Auxiliary Balanced Classifier for Class-imbalanced Semi-supervised Learningcited by 65
  32. 2020FeatMatch: Feature-Based Augmentation for Semi-Supervised Learningcited by 57
  33. 2017Smooth Neighbors on Teacher Graphs for Semi-supervised Learningcited by 48
  34. 2021CoSSL: Co-Learning of Representation and Classifier for Imbalanced Semi-Supervised Learningcited by 41
  35. 2020Milking CowMask for Semi-Supervised Image Classificationcited by 35
  36. 2018Deep Metric Transfer for Label Propagation with Limited Annotated Datacited by 33
  37. 2020Class-Imbalanced Semi-Supervised Learningcited by 28
  38. 2019Semi-Supervised Learning with Scarce Annotationscited by 25

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 11; 2024: 5; 2025: 11; 2026: 4.

34 papers.

Sort:

By area:

  1. 2023SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learningcited by 133
  2. 2022USB: A Unified Semi-supervised Learning Benchmark for Classificationcited by 95
  3. 2023Multiview Aerial Visual Recognition (MAVREC): Can Multi-view Improve Aerial Visual Perception?cited by 18
  4. 2024Temporally Consistent Koopman Autoencoders for Forecasting Dynamical Systemscited by 15
  5. 2023SemiReward: A General Reward Model for Semi-supervised Learningcited by 14
  6. 2022An Embarrassingly Simple Baseline for Imbalanced Semi-Supervised Learningcited by 12
  7. 2023SimMatchV2: Semi-Supervised Learning with Graph Consistencycited by 12
  8. 2025CustomKD: Customizing Large Vision Foundation for Edge Model Improvement via Knowledge Distillationcited by 11
  9. 2023SSB: Simple but Strong Baseline for Boosting Performance of Open-Set Semi-Supervised Learningcited by 9
  10. 2024Joint-Task Regularization for Partially Labeled Multi-Task Learningcited by 6
  11. 2022Boosting Semi-Supervised Learning with Contrastive Complementary Labelingcited by 5
  12. 2023Reliability-Adaptive Consistency Regularization for Weakly-Supervised Point Cloud Segmentationcited by 5
22 more
  1. 2023DisCo: Distilled Student Models Co-training for Semi-supervised Text Miningcited by 4
  2. 2023FAGC:Feature Augmentation on Geodesic Curve in the Pre-Shape Spacecited by 4
  3. 2024Semi-Supervised Object Detection: A Survey on Progress from CNN to Transformercited by 4
  4. 2025Semi-MoE: Mixture-of-Experts meets Semi-Supervised Histopathology Segmentationcited by 4
  5. 2025S5: Scalable Semi-Supervised Semantic Segmentation in Remote Sensingcited by 3
  6. 2025Integration of Old and New Knowledge for Generalized Intent Discovery: A Consistency-driven Prototype-Prompting Frameworkcited by 2
  7. 2025Multimodal Prototype Alignment for Semi-supervised Pathology Image Segmentationcited by 2
  8. 2023Pruning the Unlabeled Data to Improve Semi-Supervised Learningcited by 1
  9. 2024MaskMatch: Boosting Semi-Supervised Learning Through Mask Autoencoder-Driven Feature Learningcited by 1
  10. 2025RS-MTDF: Multi-Teacher Distillation and Fusion for Remote Sensing Semi-Supervised Semantic Segmentationcited by 1
  11. 2025BARL: Bilateral Alignment in Representation and Label Spaces for Semi-Supervised Volumetric Medical Image Segmentationcited by 1
  12. 2023Towards Generalizable Morph Attack Detection with Consistency Regularizationcited by 0
  13. 2023Feature Space Renormalization for Semi-supervised Learningcited by 0
  14. 2024Boosting Semi-Supervised 2D Human Pose Estimation by Revisiting Data Augmentation and Consistency Trainingcited by 0
  15. 2025SeMi: When Imbalanced Semi-Supervised Learning Meets Mining Hard Examplescited by 0
  16. 2025CRUPL: A Semi-Supervised Cyber Attack Detection with Consistency Regularization and Uncertainty-aware Pseudo-Labeling in Smart Gridcited by 0
  17. 2025Dual Teacher-Student Learning for Semi-supervised Medical Image Segmentationcited by 0
  18. 2025U-Mamba2-SSL for Semi-Supervised Tooth and Pulp Segmentation in CBCTcited by 0
  19. 2026NanoNet: Parameter-Efficient Learning with Label-Scarce Supervision for Lightweight Text Mining Modelcited by 0
  20. 2026JEPAMatch: Geometric Representation Shaping for Semi-Supervised Learningcited by 0
  21. 2026Robust Alignment: Harmonizing Clean Accuracy and Adversarial Robustness in Adversarial Trainingcited by 0
  22. 2026MTSSL: Meta-Thresholding Semi-Supervised Learningcited by 0