# X-Class: Text Classification with Extremely Weak Supervision

arXiv: https://arxiv.org/abs/2010.12794 · HTML: https://mlsanity.com/paper/2010.12794/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2020.

## Built on by (most cited first, 30 of 46)

- [ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval](https://mlsanity.com/paper/2305.10703/) (2023)
- [Weakly-supervised Text Classification Based on Keyword Graph](https://mlsanity.com/paper/2110.02591/) (2021)
- [TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal Supervision](https://mlsanity.com/paper/2403.00165/) (2024)
- [vONTSS: vMF based semi-supervised neural topic modeling with optimal transport](https://mlsanity.com/paper/2307.01226/) (2023)
- [WeDef: Weakly Supervised Backdoor Defense for Text Classification](https://mlsanity.com/paper/2205.11803/) (2022)
- [Coarse2Fine: Fine-grained Text Classification on Coarsely-grained Annotated Data](https://mlsanity.com/paper/2109.10856/) (2021)
- [LOPS: Learning Order Inspired Pseudo-Label Selection for Weakly Supervised Text Classification](https://mlsanity.com/paper/2205.12528/) (2022)
- [Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations](https://mlsanity.com/paper/2210.16637/) (2022)
- [Metadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classification](https://mlsanity.com/paper/2202.05932/) (2022)
- [Weakly Supervised Text Classification using Supervision Signals from a Language Model](https://mlsanity.com/paper/2205.06604/) (2022)
- [MotifClass: Weakly Supervised Text Classification with Higher-order Metadata Information](https://mlsanity.com/paper/2111.04022/) (2021)
- [Resolving label uncertainty with implicit posterior models](https://mlsanity.com/paper/2202.14000/) (2022)
- [Weakly-Supervised Scientific Document Classification via Retrieval-Augmented Multi-Stage Training](https://mlsanity.com/paper/2306.07193/) (2023)
- [A Survey on Retrieval And Structuring Augmented Generation with Large Language Models](https://mlsanity.com/paper/2509.10697/) (2025)
- [LIME: Weakly-Supervised Text Classification Without Seeds](https://mlsanity.com/paper/2210.06720/) (2022)
- [WOT-Class: Weakly Supervised Open-world Text Classification](https://mlsanity.com/paper/2305.12401/) (2023)
- [A Benchmark on Extremely Weakly Supervised Text Classification: Reconcile Seed Matching and Prompting Approaches](https://mlsanity.com/paper/2305.12749/) (2023)
- [PIEClass: Weakly-Supervised Text Classification with Prompting and Noise-Robust Iterative Ensemble Training](https://mlsanity.com/paper/2305.13723/) (2023)
- [Incubating Text Classifiers Following User Instruction with Nothing but LLM](https://mlsanity.com/paper/2404.10877/) (2024)
- [MEGClass: Extremely Weakly Supervised Text Classification via Mutually-Enhancing Text Granularities](https://mlsanity.com/paper/2304.01969/) (2023)
- [Weakly Supervised Multi-Label Classification of Full-Text Scientific Papers](https://mlsanity.com/paper/2306.14003/) (2023)
- [Unsupervised Dialogue Topic Segmentation in Hyperdimensional Space](https://mlsanity.com/paper/2308.10464/) (2023)
- [Less than One-shot: Named Entity Recognition via Extremely Weak Supervision](https://mlsanity.com/paper/2311.02861/) (2023)
- [Debiasing Made State-of-the-art: Revisiting the Simple Seed-based Weak Supervision for Text Classification](https://mlsanity.com/paper/2305.14794/) (2023)
- [XAI-CLASS: Explanation-Enhanced Text Classification with Extremely Weak Supervision](https://mlsanity.com/paper/2311.00189/) (2023)
- [Seed Word Selection for Weakly-Supervised Text Classification with Unsupervised Error Estimation](https://mlsanity.com/paper/2104.09765/) (2021)
- [Customer Sentiment Analysis using Weak Supervision for Customer-Agent Chat](https://mlsanity.com/paper/2111.14282/) (2021)
- [DeeperDive: The Unreasonable Effectiveness of Weak Supervision in Document Understanding A Case Study in Collaboration with UiPath Inc](https://mlsanity.com/paper/2208.08000/) (2022)
- [AX-MABSA: A Framework for Extremely Weakly Supervised Multi-label Aspect Based Sentiment Analysis](https://mlsanity.com/paper/2211.03837/) (2022)
- [Towards Weakly-Supervised Hate Speech Classification Across Datasets](https://mlsanity.com/paper/2305.02637/) (2023)

## Abstract (arXiv)

In this paper, we explore text classification with extremely weak supervision, i.e., only relying on the surface text of class names. This is a more challenging setting than the seed-driven weak supervision, which allows a few seed words per class. We opt to attack this problem from a representation learning perspective -- ideal document representations should lead to nearly the same results between clustering and the desired classification. In particular, one can classify the same corpus differently (e.g., based on topics and locations), so document representations should be adaptive to the given class names. We propose a novel framework X-Class to realize the adaptive representations. Specifically, we first estimate class representations by incrementally adding the most similar word to each class until inconsistency arises. Following a tailored mixture of class attention mechanisms, we obtain the document representation via a weighted average of contextualized word representations. With the prior of each document assigned to its nearest class, we then cluster and align the documents to classes. Finally, we pick the most confident documents from each cluster to train a text classifier. Extensive experiments demonstrate that X-Class can rival and even outperform seed-driven weakly supervised methods on 7 benchmark datasets. Our dataset and code are released at https://github.com/ZihanWangKi/XClass/ .