mlsanityData to 14 Oct 2026

SelfMatch: Combining Contrastive Self-Supervision and Consistency for Semi-Supervised Learning

arXiv 2101.06480 ↗First posted 16 Jan 2021cs.LG cs.CVcited by 35

Not on the map. Why?

Abstract (arXiv)

This paper introduces SelfMatch, a semi-supervised learning method that combines the power of contrastive self-supervised learning and consistency regularization. SelfMatch consists of two stages: (1) self-supervised pre-training based on contrastive learning and (2) semi-supervised fine-tuning based on augmentation consistency regularization. We empirically demonstrate that SelfMatch achieves the state-of-the-art results on standard benchmark datasets such as CIFAR-10 and SVHN. For example, for CIFAR-10 with 40 labeled examples, SelfMatch achieves 93.19% accuracy that outperforms the strong previous methods such as MixMatch (52.46%), UDA (70.95%), ReMixMatch (80.9%), and FixMatch (86.19%). We note that SelfMatch can close the gap between supervised learning (95.87%) and semi-supervised learning (93.19%) by using only a few labels for each class.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 25 of its 37 references that are in the map. First posted on arXiv 16 Jan 2021.

Builds on

25 of 37 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  7. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  8. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  9. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
  10. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  11. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  12. 2019Contrastive Multiview Codingcited by 1,301
13 more
  1. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  2. 2016Colorful Image Colorizationcited by 1,114
  3. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  4. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  5. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,057
  6. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 920
  7. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  8. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  9. 2019ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoringcited by 349
  10. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  11. 2019On Mutual Information Maximization for Representation Learningcited by 316
  12. 2019Revisiting Self-Supervised Visual Representation Learningcited by 251
  13. 2019Interpolation Consistency Training for Semi-Supervised Learningcited by 251

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 6; 2022: 10; 2023: 12; 2024: 4; 2025: 2; 2026: 1.

35 papers.

Sort:

  1. 2021Multi-Layer Pseudo-Supervision for Histopathology Tissue Semantic Segmentation using Patch-level Classification Labelscited by 43
  2. 2022SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generationcited by 34
  3. 2022SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for Task-Oriented Dialog Understandingcited by 28
  4. 2023Boosting Semi-Supervised Learning by Exploiting All Unlabeled Datacited by 26
  5. 2022ConMatch: Semi-Supervised Learning with Confidence-Guided Consistency Regularizationcited by 25
  6. 2022Contrastive Regularization for Semi-Supervised Learningcited by 24
  7. 2023Image Data Augmentation Approaches: A Comprehensive Survey and Future directionscited by 23
  8. 2023Rethinking Weak Supervision in Helping Contrastive Learningcited by 18
  9. 2021Barely-Supervised Learning: Semi-Supervised Learning with very few labeled imagescited by 17
  10. 2021Robust Mutual Learning for Semi-supervised Semantic Segmentationcited by 13
  11. 2021A data-centric approach for improving ambiguous labels with combined semi-supervised classification and clusteringcited by 9
  12. 2023Systematic comparison of semi-supervised and self-supervised learning for medical image classificationcited by 8
23 more
  1. 2021Unsupervised Embedding Adaptation via Early-Stage Feature Reconstruction for Few-Shot Classificationcited by 7
  2. 2021ActiveMatch: End-to-end Semi-supervised Active Representation Learningcited by 6
  3. 2022FixMatchSeg: Fixing FixMatch for Semi-Supervised Semantic Segmentationcited by 6
  4. 2022Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detectioncited by 5
  5. 2022Boosting Semi-Supervised Learning with Contrastive Complementary Labelingcited by 5
  6. 2023Graph Convolutional Networks based on Manifold Learning for Semi-Supervised Image Classificationcited by 5
  7. 2022Active Self-Semi-Supervised Learning for Few Labeled Samplescited by 4
  8. 2022AuxMix: Semi-Supervised Learning with Unconstrained Unlabeled Datacited by 4
  9. 2024Robot Instance Segmentation with Few Annotations for Graspingcited by 4
  10. 2022Deep Reference Priors: What is the best way to pretrain a model?cited by 2
  11. 2023Semi-Supervised End-To-End Contrastive Learning For Time Series Classificationcited by 2
  12. 2024Preventing Collapse in Contrastive Learning with Orthonormal Prototypes (CLOP)cited by 2
  13. 2023ContraCluster: Learning to Classify without Labels by Contrastive Self-Supervision and Prototype-Based Semi-Supervisioncited by 1
  14. 2023FireMatch: A Semi-Supervised Video Fire Detection Network Based on Consistency and Distribution Alignmentcited by 1
  15. 2024Bridging the Gaps: Utilizing Unlabeled Face Recognition Datasets to Boost Semi-Supervised Facial Expression Recognitioncited by 1
  16. 2023Shuffle & Divide: Contrastive Learning for Long Textcited by 0
  17. 2023Rethinking Scale Imbalance in Semi-supervised Object Detection for Aerial Imagescited by 0
  18. 2023On Training Implicit Meta-Learning With Applications to Inductive Weighing in Consistency Regularizationcited by 0
  19. 2023CLAF: Contrastive Learning with Augmented Features for Imbalanced Semi-Supervised Learningcited by 0
  20. 2024Smooth Pseudo-Labelingcited by 0
  21. 2025Beyond Self-Consistency: Loss-Balanced Perturbation-Based Regularization Improves Industrial-Scale Ads Rankingcited by 0
  22. 2025Semi-Supervised Contrastive Learning with Orthonormal Prototypescited by 0
  23. 2026ReCalMatch:Reliability-Calibrated Semantic Guidance for Semi-Supervised Fine-Grained Recognitioncited by 0