mlsanityData to 7 Oct 2026

Zero-Shot Text Classification with Self-Training

arXiv 2210.17541 ↗First posted 31 Oct 2022cs.CL cs.LGcited by 26

On the map: Synthetic data and self-improvement for LLMs → Self-training for few-shot NLP

Abstract (arXiv)

Recent advances in large pretrained language models have increased attention to zero-shot text classification. In particular, models finetuned on natural language inference datasets have been widely adopted as zero-shot classifiers due to their promising results and off-the-shelf availability. However, the fact that such models are unfamiliar with the target task can lead to instability and performance issues. We propose a plug-and-play method to bridge this gap using a simple self-training approach, requiring only the class names along with an unlabeled dataset, and without the need for domain expertise or trial and error. We show that fine-tuning the zero-shot classifier on its most confident predictions leads to significant performance gains across a wide range of text classification tasks, presumably since self-training adapts the zero-shot model to the task at hand.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Self-training for few-shot NLP, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 21 of its 42 references that are in the map. First posted on arXiv 31 Oct 2022.

Builds on

21 of 42 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  8. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  9. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  10. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  11. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
  12. 2019Confidence Regularized Self-Trainingcited by 418
9 more
  1. 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 299
  2. 2018Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Taskscited by 214
  3. 2021Muppet: Massive Multi-task Representations with Pre-Finetuningcited by 192
  4. 2021Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collectionscited by 128
  5. 2020Self-training Improves Pre-training for Natural Language Understandingcited by 96
  6. 2021FLEX: Unifying Evaluation for Few-Shot NLPcited by 67
  7. 2022Prompt Consistency for Zero-Shot Task Generalizationcited by 66
  8. 2021Weakly-supervised Text Classification Based on Keyword Graphcited by 25
  9. 2021Ask2Transformers: Zero-Shot Domain labelling with Pre-trained Language Modelscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 12; 2024: 7; 2025: 3; 2026: 3.

26 papers.

Sort:

By area:

  1. 2024Language Models for Text Classification: Is In-Context Learning Enough?cited by 39
  2. 2023ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrievalcited by 26
  3. 2023Don't Stop Pretraining? Make Prompt-based Fine-tuning Powerful Learnercited by 23
  4. 2023Rethinking Semi-supervised Learning with Language Modelscited by 17
  5. 2024Lessons from the Use of Natural Language Inference (NLI) in Requirements Engineering Taskscited by 12
  6. 2023Generation-driven Contrastive Self-training for Zero-shot Text Classification with Instruction-following LLMcited by 11
  7. 2023ZARA: Improving Few-Shot Self-Rationalization for Small Language Modelscited by 8
  8. 2024LLM-Generated Natural Language Meets Scaling Laws: New Explorations and Data Augmentation Methodscited by 8
  9. 2023A Simple and Effective Framework for Strict Zero-Shot Hierarchical Classificationcited by 7
  10. 2023Zero-Shot Text Classification via Self-Supervised Tuningcited by 6
  11. 2023A Benchmark on Extremely Weakly Supervised Text Classification: Reconcile Seed Matching and Prompting Approachescited by 6
  12. 2023QAID: Question Answering Inspired Few-shot Intent Detectioncited by 5
14 more
  1. 2023The Benefits of Label-Description Training for Zero-Shot Text Classificationcited by 4
  2. 2025TEXT2DB: Integration-Aware Information Extraction with Large Language Model Agentscited by 3
  3. 2022Empowering Sentence Encoders with Prompting and Label Retrieval for Zero-shot Text Classificationcited by 2
  4. 2023WC-SBERT: Zero-Shot Text Classification via SBERT with Self-Training for Wikipedia Categoriescited by 2
  5. 2024Open-world Multi-label Text Classification with Extremely Weak Supervisioncited by 2
  6. 2024Interactive Machine Teaching by Labeling Rules and Instancescited by 2
  7. 2025Text Classification Under Class Distribution Shift: A Surveycited by 2
  8. 2026Enhancing Self-Correction in Large Language Models through Multi-Perspective Reflectioncited by 2
  9. 2025Medifact at PerAnsSumm 2025: Leveraging Lightweight Models for Perspective-Specific Summarization of Clinical Q&A Forumscited by 1
  10. 2023TK-KNN: A Balanced Distance-Based Pseudo Labeling Approach for Semi-Supervised Intent Classificationcited by 0
  11. 2024Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Modelscited by 0
  12. 2024Analysis of Socially Unacceptable Discourse with Zero-shot Learningcited by 0
  13. 2026MOSLD-Bench: Multilingual Open-Set Learning and Discovery Benchmark for Text Categorizationcited by 0
  14. 2026DECSELFMASK: Leveraging Unlabeled Text via Self-Relevance-Guided Masking for Decoder-Only Classificationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective
  2. 2023Large Language Models Are Zero-Shot Text Classifiers
  3. 2026Curate-Train-Refine: A Closed-Loop Agentic Framework for Zero Shot Classification
  4. 2026Will It Zero-Shot?: Predicting Zero-Shot Classification Performance For Arbitrary Queries
  5. 2021Revisiting Self-Training for Few-Shot Learning of Language Model
  6. 2024Enhancing Low-Resource LLMs Classification with PEFT and Synthetic Data
  7. 2023Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models
  8. 2023Neural Networks Against (and For) Self-Training: Classification with Small Labeled and Large Unlabeled Sets