mlsanityData to 7 Oct 2026

Neural Data Augmentation via Example Extrapolation

arXiv 2102.01335 ↗First posted 2 Feb 2021cs.CL cs.AIcited by 54

Not on the map. Why?

Abstract (arXiv)

In many applications of machine learning, certain categories of examples may be underrepresented in the training data, causing systems to underperform on such "few-shot" cases at test time. A common remedy is to perform data augmentation, such as by duplicating underrepresented examples, or heuristically synthesizing new examples. But these remedies often fail to cover the full diversity and complexity of real examples. We propose a data augmentation approach that performs neural Example Extrapolation (Ex2). Given a handful of exemplars sampled from some distribution, Ex2 synthesizes new examples that also belong to the same distribution. The Ex2 model is learned by simulating the example generation procedure on data-rich slices of the data, and it is applied to underrepresented, few-shot slices. We apply Ex2 to a range of language understanding tasks and significantly improve over state-of-the-art methods on multiple few-shot learning benchmarks, including for relation extraction (FewRel) and intent classification + slot filling (SNIPS).

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 30 of its 36 references that are in the map. First posted on arXiv 2 Feb 2021.

Builds on

30 of 36 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  5. 2018Deep contextualized word representationscited by 2,285
  6. 2016Matching Networks for One Shot Learningcited by 2,246
  7. 2019Language Models as Knowledge Bases?cited by 1,922
  8. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  9. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  10. 2017Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning resultscited by 470
  11. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 378
  12. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 376
18 more
  1. 2017The Effectiveness of Data Augmentation in Image Classification using Deep Learningcited by 368
  2. 2019Matching the Blanks: Distributional Similarity for Relation Learningcited by 271
  3. 2018FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluationcited by 254
  4. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  5. 2019Not Enough Data? Deep Learning to the Rescue!cited by 174
  6. 2014Evaluation of Output Embeddings for Fine-Grained Image Classificationcited by 166
  7. 2019Good-Enough Compositional Data Augmentationcited by 158
  8. 2019Zero-Shot Entity Linking by Reading Entity Descriptionscited by 124
  9. 2016Data Recombination for Neural Semantic Parsingcited by 111
  10. 2019FewRel 2.0: Towards More Challenging Few-Shot Relation Classificationcited by 94
  11. 2020Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Networkcited by 83
  12. 2020Learning to Recombine and Resample Data for Compositional Generalizationcited by 65
  13. 2020Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inferencecited by 47
  14. 2019A Closer Look At Feature Space Data Augmentation For Few-Shot Intent Classificationcited by 45
  15. 2017Towards Zero-Shot Frame Semantic Parsing for Domain Scalingcited by 40
  16. 2020Learning to Classify Intents and Slot Labels Given a Handful of Examplescited by 31
  17. 2020Entity-Switched Datasets: An Approach to Auditing the In-Domain Robustness of Named Entity Recognition Modelscited by 12
  18. 2020Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learningcited by 6

Built on by

Cited by year

Citing papers per year: 2021: 11; 2022: 21; 2023: 12; 2024: 8; 2025: 2.

54 papers.

Sort:

By area:

  1. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  2. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  3. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  4. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  5. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  6. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
  7. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  8. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  9. 2021A Survey on Data Augmentation for Text Classificationcited by 111
  10. 2021Towards Zero-Label Language Learningcited by 76
  11. 2021Tailor: Generating and Perturbing Text with Semantic Controlscited by 67
  12. 2022ProGen: Progressive Zero-shot Dataset Generation via In-context Feedbackcited by 67
38 more
  1. 2022RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extractioncited by 60
  2. 2023Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustnesscited by 50
  3. 2022Data Augmentation for Intent Classification with Off-the-shelf Large Language Modelscited by 49
  4. 2022Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learningcited by 49
  5. 2022Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learningcited by 36
  6. 2021Controllable Semantic Parsing via Retrieval Augmentationcited by 35
  7. 2021STraTA: Self-Training with Task Augmentation for Better Few-shot Learningcited by 33
  8. 2022LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Taggingcited by 29
  9. 2023Propagating Knowledge Updates to LMs Through Distillationcited by 28
  10. 2021ConvFiT: Conversational Fine-Tuning of Pretrained Language Modelscited by 25
  11. 2023Selective In-Context Data Augmentation for Intent Detection using Pointwise V-Informationcited by 24
  12. 2022CLASP: Few-Shot Cross-Lingual Data Augmentation for Semantic Parsingcited by 21
  13. 2021Few-shot Controllable Style Transfer for Low-Resource Multilingual Settingscited by 20
  14. 2022Data Augmentation with Paraphrase Generation and Entity Extraction for Multimodal Dialogue Systemcited by 16
  15. 2023SQuARe: A Large-Scale Dataset of Sensitive Questions and Acceptable Responses Created Through Human-Machine Collaborationcited by 13
  16. 2024SynthesizRR: Generating Diverse Datasets with Retrieval Augmentationcited by 13
  17. 2022Multi-Vector Retrieval as Sparse Alignmentcited by 12
  18. 2021Generative Conversational Networkscited by 11
  19. 2023GDA: Generative Data Augmentation Techniques for Relation Extraction Taskscited by 10
  20. 2024mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humanscited by 10
  21. 2024Few-Shot Recalibration of Language Modelscited by 8
  22. 2024Improving Event Definition Following For Zero-Shot Event Detectioncited by 7
  23. 2023STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Modelscited by 6
  24. 2024Generate then Refine: Data Augmentation for Zero-shot Intent Detectioncited by 6
  25. 2021Good-Enough Example Extrapolationcited by 5
  26. 2022CST5: Data Augmentation for Code-Switched Semantic Parsingcited by 5
  27. 2023Fighting Bias with Bias: Promoting Model Robustness by Amplifying Dataset Biasescited by 5
  28. 2022Event Transition Planning for Open-ended Text Generationcited by 4
  29. 2024Domain Adaptation in Intent Classification Systems: A Reviewcited by 4
  30. 2021Semi-Supervised Few-Shot Intent Classification and Slot Fillingcited by 3
  31. 2022FewFedWeight: Few-shot Federated Learning Framework across Multiple NLP Taskscited by 3
  32. 2023Hate Speech Detection in Limited Data Contexts using Synthetic Data Generationcited by 3
  33. 2024CorrSynth -- A Correlated Sampling Method for Diverse Dataset Generation from LLMscited by 3
  34. 2023BERT Goes Off-Topic: Investigating the Domain Transfer Challenge using Genre Classificationcited by 2
  35. 2024On-Device Emoji Classifier Trained with GPT-based Data Augmentation for a Mobile Keyboardcited by 2
  36. 2025SNaRe: Domain-aware Data Generation for Low-Resource Event Detectioncited by 2
  37. 2022Variational Autoencoder with Disentanglement Priors for Low-Resource Task-Specific Natural Language Generationcited by 1
  38. 2022Few-shot Mining of Naturally Occurring Inputs and Outputscited by 1

50 most cited of 54 All 54 (JSON)