# Text Grafting: Near-Distribution Weak Supervision for Minority Classes in Text Classification

arXiv: https://arxiv.org/abs/2406.11115 · HTML: https://mlsanity.com/paper/2406.11115/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 51 references that are in the map.
- First posted on arXiv 17 Jun 2024.

## Builds on (most cited first, 30 of 30)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification](https://mlsanity.com/paper/2010.12421/) (2020)
- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [CLUE: A Chinese Language Understanding Evaluation Benchmark](https://mlsanity.com/paper/2004.05986/) (2020)
- [Surface Form Competition: Why the Highest Probability Answer Isn't Always Right](https://mlsanity.com/paper/2104.08315/) (2021)
- [ZeroGen: Efficient Zero-shot Learning via Dataset Generation](https://mlsanity.com/paper/2202.07922/) (2022)
- [BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization](https://mlsanity.com/paper/1906.03741/) (2019)
- [Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1512.05830/) (2015)
- [OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examples](https://mlsanity.com/paper/2307.11729/) (2023)
- [ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback](https://mlsanity.com/paper/2210.12329/) (2022)
- [Prototypical Calibration for Few-shot Learning of Language Models](https://mlsanity.com/paper/2205.10183/) (2022)
- [X-Class: Text Classification with Extremely Weak Supervision](https://mlsanity.com/paper/2010.12794/) (2020)
- [Generate, Annotate, and Learn: NLP with Synthetic Text](https://mlsanity.com/paper/2106.06168/) (2021)
- [Pre-trained Language Models Can be Fully Zero-Shot Learners](https://mlsanity.com/paper/2212.06950/) (2022)
- [A little goes a long way: Improving toxic language classification despite data scarcity](https://mlsanity.com/paper/2009.12344/) (2020)
- [LOPS: Learning Order Inspired Pseudo-Label Selection for Weakly Supervised Text Classification](https://mlsanity.com/paper/2205.12528/) (2022)
- [Supercharging Imbalanced Data Learning With Energy-based Contrastive Representation Transfer](https://mlsanity.com/paper/2011.12454/) (2020)
- [Counterfactual Data Augmentation improves Factuality of Abstractive Summarization](https://mlsanity.com/paper/2205.12416/) (2022)
- [A Benchmark on Extremely Weakly Supervised Text Classification: Reconcile Seed Matching and Prompting Approaches](https://mlsanity.com/paper/2305.12749/) (2023)
- [Incubating Text Classifiers Following User Instruction with Nothing but LLM](https://mlsanity.com/paper/2404.10877/) (2024)
- [Debiasing Made State-of-the-art: Revisiting the Simple Seed-based Weak Supervision for Text Classification](https://mlsanity.com/paper/2305.14794/) (2023)

## Built on by (most cited first, 2 of 2)

- [ConvMix: A Mixed-Criteria Data Augmentation Framework for Conversational Dense Retrieval](https://mlsanity.com/paper/2508.04001/) (2025)
- [Four Ledgers, Not One Score: Responsible Communication of LLM-Judge Calibration in Biomedical ML](https://mlsanity.com/paper/2609.15015/) (2026)

## Abstract (arXiv)

For extremely weak-supervised text classification, pioneer research generates pseudo labels by mining texts similar to the class names from the raw corpus, which may end up with very limited or even no samples for the minority classes. Recent works have started to generate the relevant texts by prompting LLMs using the class names or definitions; however, there is a high risk that LLMs cannot generate in-distribution (i.e., similar to the corpus where the text classifier will be applied) data, leading to ungeneralizable classifiers. In this paper, we combine the advantages of these two approaches and propose to bridge the gap via a novel framework, text grafting, which aims to obtain clean and near-distribution weak supervision for minority classes. Specifically, we first use LLM-based logits to mine masked templates from the raw corpus, which have a high potential for data synthesis into the target minority class. Then, the templates are filled by state-of-the-art LLMs to synthesize near-distribution texts falling into minority classes. Text grafting shows significant improvement over direct mining or synthesis on minority classes. We also use analysis and case studies to comprehend the property of text grafting.