# MultiMatch: Multihead Consistency Regularization Matching for Semi-Supervised Text Classification

arXiv: https://arxiv.org/abs/2506.07801 · HTML: https://mlsanity.com/paper/2506.07801/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 61 references that are in the map.
- First posted on arXiv 9 Jun 2025.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning](https://mlsanity.com/paper/1606.04586/) (2016)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring](https://mlsanity.com/paper/1911.09785/) (2019)
- [Identifying Mislabeled Data using the Area Under the Margin Ranking](https://mlsanity.com/paper/2001.10528/) (2020)
- [Semi-Supervised Learning with Ladder Networks](https://mlsanity.com/paper/1507.02672/) (2015)
- [MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://mlsanity.com/paper/2004.12239/) (2020)
- [CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning](https://mlsanity.com/paper/2102.09559/) (2021)
- [CoMatch: Semi-supervised Learning with Contrastive Graph Regularization](https://mlsanity.com/paper/2011.11183/) (2020)
- [Dash: Semi-Supervised Learning with Dynamic Thresholding](https://mlsanity.com/paper/2109.00650/) (2021)
- [Deep Co-Training for Semi-Supervised Image Recognition](https://mlsanity.com/paper/1803.05984/) (2018)
- [Supervised Multimodal Bitransformers for Classifying Images and Text](https://mlsanity.com/paper/1909.02950/) (2019)
- [SimMatch: Semi-supervised Learning with Similarity Matching](https://mlsanity.com/paper/2203.06915/) (2022)
- [Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning](https://mlsanity.com/paper/2007.08844/) (2020)
- [AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation](https://mlsanity.com/paper/2106.04732/) (2021)
- [Yelp Dataset Challenge: Review Rating Prediction](https://mlsanity.com/paper/1605.05362/) (2016)
- [ABC: Auxiliary Balanced Classifier for Class-imbalanced Semi-supervised Learning](https://mlsanity.com/paper/2110.10368/) (2021)
- [Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response](https://mlsanity.com/paper/2004.11838/) (2020)
- [Semi-Supervised Learning with Multi-Head Co-Training](https://mlsanity.com/paper/2107.04795/) (2021)
- [JointMatch: A Unified Approach for Diverse and Collaborative Pseudo-Labeling to Semi-Supervised Text Classification](https://mlsanity.com/paper/2310.14583/) (2023)
- [CGMatch: A Different Perspective of Semi-supervised Learning](https://mlsanity.com/paper/2503.02231/) (2025)
- [DeCrisisMB: Debiased Semi-Supervised Learning for Crisis Tweet Classification via Memory Bank](https://mlsanity.com/paper/2310.14577/) (2023)

## Built on by (most cited first, 1 of 1)

- [Semi-Supervised Learning for Large Language Models Safety and Content Moderation](https://mlsanity.com/paper/2512.21107/) (2025)

## Abstract (arXiv)

We introduce MultiMatch, a novel semi-supervised learning (SSL) algorithm combining the paradigms of co-training and consistency regularization with pseudo-labeling. At its core, MultiMatch features a pseudo-label weighting module designed for selecting and filtering pseudo-labels based on head agreement and model confidence, and weighting them according to the perceived classification difficulty. This novel module enhances and unifies three existing techniques -- heads agreement from Multihead Co-training, self-adaptive thresholds from FreeMatch, and Average Pseudo-Margins from MarginMatch -- resulting in a holistic approach that improves robustness and performance in SSL settings. Experimental results on benchmark datasets highlight the superior performance of MultiMatch, i.e., MultiMatch achieves state-of-the-art results on 8 out of 10 setups from 5 natural language processing datasets and ranks first according to the Friedman test among 21 methods. Furthermore, MultiMatch demonstrates exceptional robustness in highly imbalanced settings, outperforming the second-best approach by 3.26%, a critical advantage for real-world text classification tasks. Our code is available on GitHub.