mlsanityData to 7 Oct 2026

ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback

arXiv 2210.12329 ↗First posted 22 Oct 2022cs.CL cs.AIcited by 67

Not on the map. Why?

Abstract (arXiv)

Recently, dataset-generation-based zero-shot learning has shown promising results by training a task-specific model with a dataset synthesized from large pre-trained language models (PLMs). The final task-specific model often achieves compatible or even better performance than PLMs under the zero-shot setting, with orders of magnitude fewer parameters. However, synthetic datasets have their drawbacks. They have long been suffering from low-quality issues (e.g., low informativeness and redundancy). This explains why the massive synthetic data does not lead to better performance -- a scenario we would expect in the human-labeled data. To improve the quality of dataset synthesis, we propose a progressive zero-shot dataset generation framework, ProGen, which leverages the feedback from the task-specific model to guide the generation of new training data via in-context examples. Extensive experiments on five text classification datasets demonstrate the effectiveness of the proposed approach. We also show ProGen achieves on-par or superior performance with only 1% synthetic dataset size compared to baseline methods without in-context feedback.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Builds on 42 of its 62 references that are in the map. First posted on arXiv 22 Oct 2022.

Builds on

42 of 62 references.

References · 42

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  4. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  6. 2019Language Models as Knowledge Bases?cited by 1,922
  7. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  9. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  10. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  11. 2018Hierarchical Neural Story Generationcited by 1,182
  12. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
30 more
  1. 2019How Can We Know What Language Models Know?cited by 953
  2. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  3. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  4. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  5. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  6. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
  7. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  8. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  9. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  10. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  11. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 256
  12. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  13. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
  14. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  15. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  16. 2021Generating Datasets with Pretrained Language Modelscited by 192
  17. 2019Assessing BERT's Syntactic Abilitiescited by 191
  18. 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
  19. 2019Not Enough Data? Deep Learning to the Rescue!cited by 174
  20. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  21. 2021Noisy Channel Language Model Prompting for Few-Shot Text Classificationcited by 163
  22. 2021Meta-learning via Language Model In-context Tuningcited by 163
  23. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  24. 2020Training Question Answering Models From Synthetic Datacited by 95
  25. 2021Towards Zero-Label Language Learningcited by 76
  26. 2021Neural Data Augmentation via Example Extrapolationcited by 54
  27. 2021Reordering Examples Helps during Priming-based Few-Shot Learningcited by 50
  28. 2020Generative Data Augmentation for Commonsense Reasoningcited by 49
  29. 2021Controllable Semantic Parsing via Retrieval Augmentationcited by 35
  30. 2020DQI: Measuring Data Quality in NLPcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 18; 2024: 36; 2025: 12; 2026: 1.

67 papers.

Sort:

By area:

  1. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 240
  2. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 230
  3. 2023G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Modelcited by 208
  4. 2023Compositional Exemplars for In-context Learningcited by 143
  5. 2023DetGPT: Detect What You Need via Reasoningcited by 112
  6. 2023Finding Support Examples for In-Context Learningcited by 104
  7. 2023VITATECS: A Diagnostic Dataset for Temporal Concept Understanding of Video-Language Modelscited by 55
  8. 2023OpenICL: An Open-Source Framework for In-context Learningcited by 51
  9. 2024Forcing Diffuse Distributions out of Language Modelscited by 48
  10. 2023Small Models are Valuable Plug-ins for Large Language Modelscited by 42
  11. 2024Better Synthetic Data by Retrieving and Transforming Existing Datasetscited by 36
  12. 2024Generative AI for Synthetic Data Generation: Methods, Challenges and the Futurecited by 31
38 more
  1. 2024Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Modelscited by 31
  2. 2023Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Modelscited by 30
  3. 2024CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibrationcited by 30
  4. 2023Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Generationcited by 27
  5. 2023ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrievalcited by 26
  6. 2023Contrastive Learning of Sentence Embeddings from Scratchcited by 24
  7. 2023Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Modelscited by 20
  8. 2024Active Data Curation Effectively Distills Large-Scale Multimodal Modelscited by 19
  9. 2023TarGEN: Targeted Data Generation with Large Language Modelscited by 17
  10. 2024ToxiCraft: A Novel Framework for Synthetic Generation of Harmful Informationcited by 17
  11. 2023FreeAL: Towards Human-Free Active Learning in the Era of Large Language Modelscited by 14
  12. 2024SynthesizRR: Generating Diverse Datasets with Retrieval Augmentationcited by 13
  13. 2024textitTrans-LoRA: towards data-free Transferable Parameter Efficient Finetuningcited by 13
  14. 2023Generating Data for Symbolic Language with Large Language Modelscited by 12
  15. 2024The ALCHEmist: Automated Labeling 500x CHEaper Than LLM Data Annotatorscited by 10
  16. 2024DiffLM: Controllable Synthetic Data Generation via Diffusion Language Modelscited by 9
  17. 2025Few-shot LLM Synthetic Data with Distribution Matchingcited by 9
  18. 2024FastGAS: Fast Graph-based Annotation Selection for In-Context Learningcited by 8
  19. 2023Fabricator: An Open Source Toolkit for Generating Labeled Training Data with Teacher LLMscited by 7
  20. 2023Deeper Understanding of Black-box Predictions via Generalized Influence Functionscited by 7
  21. 2024Hybrid Training Approaches for LLMs: Leveraging Real and Synthetic Data to Enhance Model Performance in Domain-Specific Applicationscited by 7
  22. 2024Incubating Text Classifiers Following User Instruction with Nothing but LLMcited by 6
  23. 2024ToxiLab: How Well Do Open-Source LLMs Generate Synthetic Toxicity Data?cited by 5
  24. 2025Mutual Reinforcement of LLM Dialogue Synthesis and Summarization Capabilities for Few-Shot Dialogue Summarizationcited by 5
  25. 2024Unveiling the Achilles' Heel of NLG Evaluators: A Unified Adversarial Framework Driven by Large Language Modelscited by 4
  26. 2024On The Persona-based Summarization of Domain-Specific Documentscited by 4
  27. 2024FuseGen: PLM Fusion for Data-generation based Zero-shot Learningcited by 4
  28. 2025Zero-Shot and Efficient Clarification Need Prediction in Conversational Searchcited by 4
  29. 2025Towards Harnessing the Collaborative Power of Large and Small Models for Domain Taskscited by 4
  30. 2024Improving Sentence Embeddings with Automatic Generation of Training Data Using Few-shot Examplescited by 3
  31. 2024UniGen: Universal Domain Generalization for Sentiment Classification via Zero-shot Dataset Generationcited by 3
  32. 2024Training Task Experts through Retrieval Based Distillationcited by 3
  33. 2024Controlled Automatic Task-Specific Synthetic Data Generation for Hallucination Detectioncited by 3
  34. 2024CorrSynth -- A Correlated Sampling Method for Diverse Dataset Generation from LLMscited by 3
  35. 2024Text Grafting: Near-Distribution Weak Supervision for Minority Classes in Text Classificationcited by 2
  36. 2024Large Visual-Language Models Are Also Good Classifiers: A Study of In-Context Multimodal Fake News Detectioncited by 2
  37. 2024RED-CT: A Systems Design Methodology for Using LLM-labeled Data to Train and Deploy Edge Classifiers for Computational Social Sciencecited by 2
  38. 2024AIDE: Attribute-Guided MultI-Hop Data Expansion for Data Scarcity in Task-Specific Fine-tuningcited by 2

50 most cited of 67 All 67 (JSON)