mlsanityData to 14 Oct 2026

ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval

arXiv 2305.10703 ↗First posted 18 May 2023cs.CL cs.IR cs.LGcited by 26

Not on the map. Why?

Abstract (arXiv)

With the development of large language models (LLMs), zero-shot learning has attracted much attention for various NLP tasks. Different from prior works that generate training data with billion-scale natural language generation (NLG) models, we propose a retrieval-enhanced framework to create training data from a general-domain unlabeled corpus. To realize this, we first conduct contrastive pretraining to learn an unsupervised dense retriever for extracting the most relevant documents using class-descriptive verbalizers. We then further propose two simple strategies, namely Verbalizer Augmentation with Demonstrations and Self-consistency Guided Filtering to improve the topic coverage of the dataset while removing noisy examples. Experiments on nine datasets demonstrate that REGEN achieves 4.3% gain over the strongest baselines and saves around 70% of the time compared to baselines using large NLG models. Besides, REGEN can be naturally integrated with recently proposed large language models to boost performance.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 74 of its 103 references that are in the map. First posted on arXiv 18 May 2023.

Builds on

74 of 103 references.

References · 74

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  9. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  10. 2017Billion-scale similarity search with GPUscited by 2,497
  11. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  12. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
38 more
  1. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  2. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  3. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  4. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  5. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  6. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  7. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  8. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  9. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  10. 2019When Does Label Smoothing Help?cited by 988
  11. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  12. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
  13. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  14. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  15. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  16. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  17. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
  18. 2019Defending Against Neural Fake Newscited by 559
  19. 2016Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filteringcited by 520
  20. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
  21. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  22. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
  23. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 353
  24. 2021True Few-Shot Learning with Language Modelscited by 344
  25. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 337
  26. 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 308
  27. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 211
  28. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  29. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  30. 2021Generating Datasets with Pretrained Language Modelscited by 193
  31. 2021Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classificationcited by 183
  32. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  33. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
  34. 2020Cold-start Active Learning through Self-supervised Language Modelingcited by 128
  35. 2021Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collectionscited by 128
  36. 2019Learning Dense Representations for Entity Retrievalcited by 122
  37. 2021Improving and Simplifying Pattern Exploiting Trainingcited by 97
  38. 2021WRENCH: A Comprehensive Benchmark for Weak Supervisioncited by 96

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 13; 2025: 6; 2026: 1.

26 papers.

Sort:

By area:

  1. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  2. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 232
  3. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Codecited by 61
  4. 2024Generative AI for Synthetic Data Generation: Methods, Challenges and the Futurecited by 33
  5. 2023TarGEN: Targeted Data Generation with Large Language Modelscited by 17
  6. 2024Retrieval-style In-Context Learning for Few-shot Hierarchical Text Classificationcited by 16
  7. 2024SynthesizRR: Generating Diverse Datasets with Retrieval Augmentationcited by 13
  8. 2023DyGen: Learning from Noisy Labels via Dynamics-Enhanced Generative Modelingcited by 10
  9. 2023It's All Relative! -- A Synthetic Query Generation Approach for Improving Zero-Shot Relevance Predictioncited by 10
  10. 2025Few-shot LLM Synthetic Data with Distribution Matchingcited by 10
  11. 2024Boosting Reward Model with Preference-Conditional Multi-Aspect Synthetic Data Generationcited by 9
  12. 2024A Little Human Data Goes A Long Waycited by 6
14 more
  1. 2024Retrieval Augmented Zero-Shot Text Classificationcited by 4
  2. 2024Training Task Experts through Retrieval Based Distillationcited by 3
  3. 2024CorrSynth -- A Correlated Sampling Method for Diverse Dataset Generation from LLMscited by 3
  4. 2025Advances in Pre-trained Language Models for Domain-Specific Text Classification: A Systematic Reviewcited by 3
  5. 2025Text Classification Under Class Distribution Shift: A Surveycited by 2
  6. 2025Does Prompt Design Impact Quality of Data Imputation by LLMs?cited by 2
  7. 2023XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learnerscited by 1
  8. 2024ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Modelscited by 1
  9. 2024Clustering Algorithms and RAG Enhancing Semi-Supervised Text Classification with Large LLMscited by 1
  10. 2023Local Boosting for Weakly-Supervised Learningcited by 0
  11. 2024Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Modelscited by 0
  12. 2024JAPAGEN: Efficient Few/Zero-shot Learning via Japanese Training Dataset Generation with LLMcited by 0
  13. 2025Batch Aggregation: An Approach to Enhance Text Classification with Correlated Augmented Datacited by 0
  14. 2026MOSLD-Bench: Multilingual Open-Set Learning and Discovery Benchmark for Text Categorizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective
  2. 2023Large Language Models Are Zero-Shot Text Classifiers
  3. 2026Curate-Train-Refine: A Closed-Loop Agentic Framework for Zero Shot Classification
  4. 2023Zero-shot Retrieval: Augmenting Pre-trained Models with Search Engines
  5. 2023Generation-driven Contrastive Self-training for Zero-shot Text Classification with Instruction-following LLM
  6. 2026ZeroUnlearn: Few-Shot Knowledge Unlearning in Large Language Models
  7. 2021Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models
  8. 2023Instruction Distillation Makes Large Language Models Efficient Zero-shot Rankers