# MEGClass: Extremely Weakly Supervised Text Classification via Mutually-Enhancing Text Granularities

arXiv: https://arxiv.org/abs/2304.01969 · HTML: https://mlsanity.com/paper/2304.01969/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 43 references that are in the map.
- First posted on arXiv 4 Apr 2023.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [Unsupervised Domain Clusters in Pretrained Language Models](https://mlsanity.com/paper/2004.02105/) (2020)
- [Weakly-Supervised Neural Text Classification](https://mlsanity.com/paper/1809.01478/) (2018)
- [X-Class: Text Classification with Extremely Weak Supervision](https://mlsanity.com/paper/2010.12794/) (2020)
- [Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations](https://mlsanity.com/paper/2202.04582/) (2022)
- [Discriminative Topic Mining via Category-Name Guided Text Embedding](https://mlsanity.com/paper/1908.07162/) (2019)
- [Weakly-supervised Text Classification Based on Keyword Graph](https://mlsanity.com/paper/2110.02591/) (2021)
- [Learning with Weak Supervision for Email Intent Detection](https://mlsanity.com/paper/2005.13084/) (2020)
- [Effective Seed-Guided Topic Discovery by Integrating Multiple Types of Contexts](https://mlsanity.com/paper/2212.06002/) (2022)
- [Weakly Supervised Text Classification using Supervision Signals from a Language Model](https://mlsanity.com/paper/2205.06604/) (2022)
- [Unsupervised Key Event Detection from Massive Text Corpora](https://mlsanity.com/paper/2206.04153/) (2022)
- [PDSum: Prototype-driven Continuous Summarization of Evolving Multi-document Sets Stream](https://mlsanity.com/paper/2302.05550/) (2023)

## Built on by (most cited first, 4 of 4)

- [TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal Supervision](https://mlsanity.com/paper/2403.00165/) (2024)
- [PIEClass: Weakly-Supervised Text Classification with Prompting and Noise-Robust Iterative Ensemble Training](https://mlsanity.com/paper/2305.13723/) (2023)
- [Synergizing Unsupervised Episode Detection with LLMs for Large-Scale News Events](https://mlsanity.com/paper/2408.04873/) (2024)
- [Don't Pay Attention, PLANT It: Pretraining Attention via Learning-to-Rank](https://mlsanity.com/paper/2410.23066/) (2024)

## Abstract (arXiv)

Text classification is essential for organizing unstructured text. Traditional methods rely on human annotations or, more recently, a set of class seed words for supervision, which can be costly, particularly for specialized or emerging domains. To address this, using class surface names alone as extremely weak supervision has been proposed. However, existing approaches treat different levels of text granularity (documents, sentences, or words) independently, disregarding inter-granularity class disagreements and the context identifiable exclusively through joint extraction. In order to tackle these issues, we introduce MEGClass, an extremely weakly-supervised text classification method that leverages Mutually-Enhancing Text Granularities. MEGClass utilizes coarse- and fine-grained context signals obtained by jointly considering a document's most class-indicative words and sentences. This approach enables the learning of a contextualized document representation that captures the most discriminative class indicators. By preserving the heterogeneity of potential classes, MEGClass can select the most informative class-indicative documents as iterative feedback to enhance the initial word-based class representations and ultimately fine-tune a pre-trained text classifier. Extensive experiments on seven benchmark datasets demonstrate that MEGClass outperforms other weakly and extremely weakly supervised methods.