mlsanityData to 14 Oct 2026

Improving Short Text Classification With Augmented Data Using GPT-3

arXiv 2205.10981 ↗First posted 23 May 2022cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

GPT-3 is a large-scale natural language model developed by OpenAI that can perform many different tasks, including topic classification. Although researchers claim that it requires only a small number of in-context examples to learn a task, in practice GPT-3 requires these training examples to be either of exceptional quality or a higher quantity than easily created by hand. To address this issue, this study teaches GPT-3 to classify whether a question is related to data science by augmenting a small training set with additional examples generated by GPT-3 itself. This study compares two classifiers: the GPT-3 Classification Endpoint with augmented examples, and the GPT-3 Completion Endpoint with an optimal training set chosen using a genetic algorithm. We find that while the augmented Completion Endpoint achieves upwards of 80 percent validation accuracy, using the augmented Classification Endpoint yields more consistent accuracy on unseen examples. In this way, giving large-scale machine learning models like GPT-3 the ability to propose their own additional training examples can result in improved classification performance.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 30 of its 122 references that are in the map. First posted on arXiv 23 May 2022.

Builds on

30 of 122 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  7. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  8. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  9. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  10. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  11. 2017Deep Learning Scaling is Predictable, Empiricallycited by 582
  12. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
18 more
  1. 2021A Survey of Data Augmentation Approaches for NLPcited by 409
  2. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 326
  3. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 260
  4. 2018Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relationscited by 201
  5. 2015Semi-supervised Sequence Learningcited by 198
  6. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 187
  7. 2019Not Enough Data? Deep Learning to the Rescue!cited by 175
  8. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  9. 2019Text Classification Algorithms: A Surveycited by 163
  10. 2021Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challengescited by 145
  11. 2020Deep Learning Based Text Classification: A Comprehensive Reviewcited by 141
  12. 2020Self-training Improves Pre-training for Natural Language Understandingcited by 96
  13. 2019DocBERT: BERT for Document Classificationcited by 68
  14. 2020Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generationcited by 47
  15. 2020CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understandingcited by 41
  16. 2022METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signalscited by 20
  17. 2021Understanding Emails and Drafting Responses -- An Approach Using GPT-3cited by 10
  18. 2021Genetic Algorithms For Extractive Summarizationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 5; 2026: 1.

11 papers.

  1. 2023Generative Pre-trained Transformer: A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directionscited by 55
  2. 2024A Comprehensive Survey on Data Augmentationcited by 40
  3. 2023The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Taskscited by 35
  4. 2024LLMAEL: Large Language Models are Good Context Augmenters for Entity Linkingcited by 19
  5. 2024LLM-Generated Natural Language Meets Scaling Laws: New Explorations and Data Augmentation Methodscited by 8
  6. 2025Backtranslation and paraphrasing in the LLM era? Comparing data augmentation methods for emotion classificationcited by 2
  7. 2025CIEGAD: Cluster-Conditioned Interpolative and Extrapolative Framework for Geometry-Aware and Domain-Aligned Data Augmentationcited by 1
  8. 2025Weighted Multi-Prompt Learning with Description-free Large Language Model Distillationcited by 0
  9. 2025Online Anti-sexist Speech: Identifying Resistance to Gender Bias in Political Discoursecited by 0
  10. 2025Optimizing Small Transformer-Based Language Models for Multi-Label Sentiment Analysis in Short Textscited by 0
  11. 2026GRaSp: Automatic Example Optimization for In-Context Learning in Low-Data Taskscited by 0