# Progressive Class Semantic Matching for Semi-supervised Text Classification

arXiv: https://arxiv.org/abs/2205.10189 · HTML: https://mlsanity.com/paper/2205.10189/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 41 references that are in the map.
- First posted on arXiv 20 May 2022.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring](https://mlsanity.com/paper/1911.09785/) (2019)
- [BAE: BERT-based Adversarial Examples for Text Classification](https://mlsanity.com/paper/2004.01970/) (2020)
- [Adversarial Training Methods for Semi-Supervised Text Classification](https://mlsanity.com/paper/1605.07725/) (2016)
- [Interpolation Consistency Training for Semi-Supervised Learning](https://mlsanity.com/paper/1903.03825/) (2019)
- [MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://mlsanity.com/paper/2004.12239/) (2020)
- [Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence](https://mlsanity.com/paper/1903.09588/) (2019)
- [Learning by Transduction](https://mlsanity.com/paper/1301.7375/) (2013)
- [Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks](https://mlsanity.com/paper/1511.06390/) (2015)
- [Semi-Supervised Sequence Modeling with Cross-View Training](https://mlsanity.com/paper/1809.08370/) (2018)
- [To BERT or Not to BERT: Comparing Task-specific and Task-agnostic Semi-Supervised Approaches for Sequence Tagging](https://mlsanity.com/paper/2010.14042/) (2020)
- [Neural Semi-supervised Learning for Text Classification Under Large-Scale Pretraining](https://mlsanity.com/paper/2011.08626/) (2020)
- [FLiText: A Faster and Lighter Semi-Supervised Text Classification with Convolution Networks](https://mlsanity.com/paper/2110.11869/) (2021)

## Built on by (most cited first, 7 of 7)

- [JointMatch: A Unified Approach for Diverse and Collaborative Pseudo-Labeling to Semi-Supervised Text Classification](https://mlsanity.com/paper/2310.14583/) (2023)
- [DisCo: Distilled Student Models Co-training for Semi-supervised Text Mining](https://mlsanity.com/paper/2305.12074/) (2023)
- [IDoFew: Intermediate Training Using Dual-Clustering in Language Models for Few Labels Text Classification](https://mlsanity.com/paper/2401.04025/) (2024)
- [Labels Need Prompts Too: Mask Matching for Natural Language Understanding Tasks](https://mlsanity.com/paper/2312.08726/) (2023)
- [Lightweight Contenders: Navigating Semi-Supervised Text Mining through Peer Collaboration and Self Transcendence](https://mlsanity.com/paper/2412.00883/) (2024)
- [Progressive Feature Adjustment for Semi-supervised Learning from Pretrained Models](https://mlsanity.com/paper/2309.04659/) (2023)
- [Semi-Supervised Learning with Balanced Deep Representation Distributions](https://mlsanity.com/paper/2603.21056/) (2026)

## Abstract (arXiv)

Semi-supervised learning is a promising way to reduce the annotation cost for text-classification. Combining with pre-trained language models (PLMs), e.g., BERT, recent semi-supervised learning methods achieved impressive performance. In this work, we further investigate the marriage between semi-supervised learning and a pre-trained language model. Unlike existing approaches that utilize PLMs only for model parameter initialization, we explore the inherent topic matching capability inside PLMs for building a more powerful semi-supervised learning approach. Specifically, we propose a joint semi-supervised learning process that can progressively build a standard K-way classifier and a matching network for the input text and the Class Semantic Representation (CSR). The CSR will be initialized from the given labeled sentences and progressively updated through the training process. By means of extensive experiments, we show that our method can not only bring remarkable improvement to baselines, but also overall be more stable, and achieves state-of-the-art performance in semi-supervised text classification.