mlsanityData to 7 Oct 2026

NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation

arXiv 2112.02721 ↗First posted 6 Dec 2021cs.CL cs.AI cs.LGcited by 72

Not on the map. Why?

Abstract (arXiv)

Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained on. In this paper, we present NL-Augmenter, a new participatory Python-based natural language augmentation framework which supports the creation of both transformations (modifications to the data) and filters (data splits according to specific features). We describe the framework and an initial set of 117 transformations and 23 filters for a variety of natural language tasks. We demonstrate the efficacy of NL-Augmenter by using several of its transformations to analyze the robustness of popular natural language models. The infrastructure, datacards and robustness analysis results are available publicly on the NL-Augmenter repository (https://github.com/GEM-benchmark/NL-Augmenter).

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). Builds on 77 of its 143 references that are in the map. First posted on arXiv 6 Dec 2021.

Builds on

77 of 143 references.

References · 77

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  5. 2018Model Cards for Model Reportingcited by 1,068
  6. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  7. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  8. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  9. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  10. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  11. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  12. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
38 more
  1. 2017Word Translation Without Parallel Datacited by 480
  2. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  3. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  4. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  5. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 395
  6. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 299
  7. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  8. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 219
  9. 2018Gender Bias in Neural Natural Language Processingcited by 210
  10. 2020Reformulating Unsupervised Style Transfer as Paraphrase Generationcited by 181
  11. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  12. 2019Simple BERT Models for Relation Extraction and Semantic Role Labelingcited by 154
  13. 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152
  14. 2018Higher-order Coreference Resolution with Coarse-to-fine Inferencecited by 150
  15. 2017Data Augmentation for Low-Resource Neural Machine Translationcited by 139
  16. 2020Contextualized Perturbation for Textual Adversarial Attackcited by 136
  17. 2016Sequence-to-Sequence Learning as Beam-Search Optimizationcited by 126
  18. 2021Robustness Gym: Unifying the NLP Evaluation Landscapecited by 107
  19. 2021An Empirical Survey of Data Augmentation for Limited Data Learning in NLPcited by 101
  20. 2019Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systemscited by 99
  21. 2020CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLPcited by 79
  22. 2020An Analysis of Simple Data Augmentation for Named Entity Recognitioncited by 75
  23. 2020It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbationscited by 64
  24. 2019UER: An Open-Source Toolkit for Pre-training Modelscited by 61
  25. 2020Neural Syntactic Preordering for Controlled Paraphrase Generationcited by 56
  26. 2018Content preserving text generation with attribute controlscited by 47
  27. 2019Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsingcited by 46
  28. 2017ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translationscited by 44
  29. 2021They, Them, Theirs: Rewriting with Gender-Neutral Englishcited by 44
  30. 2019Attention-Informed Mixed-Language Training for Zero-shot Cross-lingual Task-oriented Dialogue Systemscited by 42
  31. 2020Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuningcited by 39
  32. 2020From Hero to Z'eroe: A Benchmark of Low-Level Adversarial Attackscited by 36
  33. 2019Improving short text classification through global augmentation methodscited by 33
  34. 2021Continual Mixed-Language Pre-Training for Extremely Low-Resource Neural Machine Translationcited by 29
  35. 2016emoji2vec: Learning Emoji Representations from their Descriptioncited by 28
  36. 2017Learning Paraphrastic Sentence Embeddings from Back-Translated Bitextcited by 27
  37. 2021Substructure Substitution: Structured Data Augmentation for NLPcited by 25
  38. 2021NoiseQA: Challenge Set Evaluation for User-Centric Question Answeringcited by 25

50 most cited of 77 All 77 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 24; 2023: 19; 2024: 19; 2025: 6; 2026: 3.

72 papers.

Sort:

By area:

  1. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  2. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  3. 2024Generative Representational Instruction Tuningcited by 227
  4. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 178
  5. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 174
  6. 2022Multi-lingual Evaluation of Code Generation Modelscited by 158
  7. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 154
  8. 2023Hallucinations in Large Multilingual Translation Modelscited by 126
  9. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  10. 2024LLM Dataset Inference: Did you train on my dataset?cited by 107
  11. 2022ReCode: Robustness Evaluation of Code Generation Modelscited by 100
  12. 2023NLPositionality: Characterizing Design Biases of Datasets and Modelscited by 91
38 more
  1. 2023Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicitycited by 86
  2. 2023FinGPT: Large Generative Models for a Small Languagecited by 49
  3. 2024garak: A Framework for Security Probing Large Language Modelscited by 48
  4. 2022ORCA: A Challenging Benchmark for Arabic Language Understandingcited by 46
  5. 2022Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionalitycited by 44
  6. 2022NusaCrowd: Open Source Initiative for Indonesian NLP Resourcescited by 38
  7. 2022GEMv2: Multilingual NLG Benchmarking in a Single Line of Codecited by 36
  8. 2022Robustness Analysis of Video-Language Models Against Visual and Language Perturbationscited by 23
  9. 2024Benchmarking Large Multimodal Models against Common Corruptionscited by 22
  10. 2024GenQREnsemble: Zero-Shot LLM Ensemble Prompting for Generative Query Reformulationcited by 22
  11. 2024IRCoder: Intermediate Representations Make Language Models Robust Multilingual Code Generatorscited by 21
  12. 2025A Study of LLMs' Preferences for Libraries and Programming Languagescited by 20
  13. 2022CausalAgents: A Robustness Benchmark for Motion Forecasting using Causal Relationshipscited by 18
  14. 2024From One to Many: Expanding the Scope of Toxicity Mitigation in Language Modelscited by 18
  15. 2023Bring Your Own Data! Self-Supervised Evaluation for Large Language Modelscited by 17
  16. 2022How Many Data Samples is an Additional Instruction Worth?cited by 16
  17. 2022Intriguing Properties of Compression on Multilingual Modelscited by 15
  18. 2022Learning Disentangled Textual Representations via Statistical Measures of Similaritycited by 14
  19. 2023CLEVA: Chinese Language Models EVAluation Platformcited by 13
  20. 2023HyperTab: Hypernetwork Approach for Deep Learning on Small Tabular Datasetscited by 10
  21. 2022PreQuEL: Quality Estimation of Machine Translation Outputs in Advancecited by 9
  22. 2023Beyond Testers' Biases: Guiding Model Testing with Knowledge Bases using LLMscited by 9
  23. 2024Token Alignment via Character Matching for Subword Completioncited by 9
  24. 2022DataLab: A Platform for Data Analysis and Interventioncited by 8
  25. 2022Can Question Rewriting Help Conversational Question Answering?cited by 8
  26. 2025PredictaBoard: Benchmarking LLM Score Predictabilitycited by 7
  27. 2023An Interactive Query Generation Assistant using LLM-based Prompt Modification and User Feedbackcited by 6
  28. 2024DARE: Diverse Visual Question Answering with Robustness Evaluationcited by 6
  29. 2022Every picture tells a story: Image-grounded controllable stylistic story generationcited by 5
  30. 2022Can Pretrained Language Models (Yet) Reason Deductively?cited by 5
  31. 2022EUREKA: EUphemism Recognition Enhanced through Knn-based methods and Augmentationcited by 5
  32. 2022AugCSE: Contrastive Sentence Embedding with Diverse Augmentationscited by 5
  33. 2023Is AI Art Another Industrial Revolution in the Making?cited by 5
  34. 2023A Methodology for Generative Spelling Correction via Natural Spelling Errors Emulation across Multiple Domains and Languagescited by 5
  35. 2023IndoRobusta: Towards Robustness Against Diverse Code-Mixed Indonesian Local Languagescited by 5
  36. 2024Generative Query Reformulation Using Ensemble Prompting, Document Fusion, and Relevance Feedbackcited by 5
  37. 2022MABSplit: Faster Forest Training Using Multi-Armed Banditscited by 4
  38. 2023Greener yet Powerful: Taming Large Code Generation Models with Quantizationcited by 4

50 most cited of 72 All 72 (JSON)