# ConMatch: Semi-Supervised Learning with Confidence-Guided Consistency Regularization

arXiv: https://arxiv.org/abs/2208.08631 · HTML: https://mlsanity.com/paper/2208.08631/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 60 references that are in the map.
- First posted on arXiv 18 Aug 2022.

## Builds on (most cited first, 30 of 43)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning](https://mlsanity.com/paper/1606.04586/) (2016)
- [DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition](https://mlsanity.com/paper/1310.1531/) (2013)
- [Meta Pseudo Labels](https://mlsanity.com/paper/2003.10580/) (2020)
- [Rethinking Pre-training and Self-training](https://mlsanity.com/paper/2006.06882/) (2020)
- [ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring](https://mlsanity.com/paper/1911.09785/) (2019)
- [S4L: Self-Supervised Semi-Supervised Learning](https://mlsanity.com/paper/1905.03670/) (2019)
- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)

## Built on by (most cited first, 25 of 25)

- [SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learning](https://mlsanity.com/paper/2301.10921/) (2023)
- [Semi-supervised learning made simple with self-supervised clustering](https://mlsanity.com/paper/2306.07483/) (2023)
- [Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learning](https://mlsanity.com/paper/2308.06777/) (2023)
- [Neighborhood-Regularized Self-Training for Learning with Few Labels](https://mlsanity.com/paper/2301.03726/) (2023)
- [SemiReward: A General Reward Model for Semi-supervised Learning](https://mlsanity.com/paper/2310.03013/) (2023)
- [An Embarrassingly Simple Baseline for Imbalanced Semi-Supervised Learning](https://mlsanity.com/paper/2211.11086/) (2022)
- [Semi-supervised Semantic Segmentation with Multi-Constraint Consistency Learning](https://mlsanity.com/paper/2503.17914/) (2025)
- [Generalized Category Discovery via Reciprocal Learning and Class-Wise Distribution Regularization](https://mlsanity.com/paper/2506.02334/) (2025)
- [When Confidence Fails: Revisiting Pseudo-Label Selection in Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2509.16704/) (2025)
- [PacGDC: Label-Efficient Generalizable Depth Completion with Projection Ambiguity and Consistency](https://mlsanity.com/paper/2507.07374/) (2025)
- [OmniGaze: Reward-inspired Generalizable Gaze Estimation In The Wild](https://mlsanity.com/paper/2510.13660/) (2025)
- [Bidirectional Channel-selective Semantic Interaction for Semi-Supervised Medical Segmentation](https://mlsanity.com/paper/2601.05855/) (2026)
- [Scaling Up Semi-supervised Learning with Unconstrained Unlabelled Data](https://mlsanity.com/paper/2306.01222/) (2023)
- [Improving 3D Semi-supervised Learning by Effectively Utilizing All Unlabelled Data](https://mlsanity.com/paper/2409.13977/) (2024)
- [SPACE: SPAtial-aware Consistency rEgularization for anomaly detection in Industrial applications](https://mlsanity.com/paper/2411.05822/) (2024)
- [A Method of Moments Embedding Constraint and its Application to Semi-Supervised Learning](https://mlsanity.com/paper/2404.17978/) (2024)
- [Semi-supervised 2D Human Pose Estimation via Adaptive Keypoint Masking](https://mlsanity.com/paper/2404.14835/) (2024)
- [Smooth Pseudo-Labeling](https://mlsanity.com/paper/2405.14313/) (2024)
- [Exploiting Minority Pseudo-Labels for Semi-Supervised Fine-grained Road Scene Understanding](https://mlsanity.com/paper/2409.12680/) (2024)
- [CoVLM: Leveraging Consensus from Vision-Language Models for Semi-supervised Multi-modal Fake News Detection](https://mlsanity.com/paper/2410.04426/) (2024)
- [GTPC-SSCD: Gate-guided Two-level Perturbation Consistency-based Semi-Supervised Change Detection](https://mlsanity.com/paper/2411.18880/) (2024)
- [CAD: Confidence-Aware Adaptive Displacement for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2502.00536/) (2025)
- [Beyond Self-Consistency: Loss-Balanced Perturbation-Based Regularization Improves Industrial-Scale Ads Ranking](https://mlsanity.com/paper/2502.18478/) (2025)
- [Context Consistency Learning via Sentence Removal for Semi-Supervised Video Paragraph Grounding](https://mlsanity.com/paper/2506.18476/) (2025)
- [CoVar: Confidence-Variance-Guided Pseudo-Label Selection for Semi-Supervised Learning](https://mlsanity.com/paper/2601.11670/) (2026)

## Abstract (arXiv)

We present a novel semi-supervised learning framework that intelligently leverages the consistency regularization between the model's predictions from two strongly-augmented views of an image, weighted by a confidence of pseudo-label, dubbed ConMatch. While the latest semi-supervised learning methods use weakly- and strongly-augmented views of an image to define a directional consistency loss, how to define such direction for the consistency regularization between two strongly-augmented views remains unexplored. To account for this, we present novel confidence measures for pseudo-labels from strongly-augmented views by means of weakly-augmented view as an anchor in non-parametric and parametric approaches. Especially, in parametric approach, we present, for the first time, to learn the confidence of pseudo-label within the networks, which is learned with backbone model in an end-to-end manner. In addition, we also present a stage-wise training to boost the convergence of training. When incorporated in existing semi-supervised learners, ConMatch consistently boosts the performance. We conduct experiments to demonstrate the effectiveness of our ConMatch over the latest methods and provide extensive ablation studies. Code has been made publicly available at https://github.com/JiwonCocoder/ConMatch.