mlsanityData to 7 Oct 2026

How Many Data Samples is an Additional Instruction Worth?

arXiv 2203.09161 ↗First posted 17 Mar 2022cs.CL cs.AI cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

Recently introduced instruction-paradigm empowers non-expert users to leverage NLP resources by defining a new task in natural language. Instruction-tuned models have significantly outperformed multitask learning models (without instruction); however they are far from state-of-the-art task-specific models. Conventional approaches to improve model performance via creating datasets with large number of task instances or architectural changes in the model may not be feasible for non-expert users. However, they can write alternate instructions to represent an instruction task. Is Instruction-augmentation helpful? We augment a subset of tasks in the expanded version of NATURAL INSTRUCTIONS with additional instructions and find that it significantly improves model performance (up to 35%), especially in the low-data regime. Our results indicate that an additional instruction can be equivalent to ~200 data samples on average across tasks.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 39 of its 58 references that are in the map. First posted on arXiv 17 Mar 2022.

Builds on

39 of 58 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  10. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  11. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  12. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
27 more
  1. 2019How Can We Know What Language Models Know?cited by 953
  2. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  3. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  4. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  5. 2021MetaICL: Learning to Learn In Contextcited by 450
  6. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 423
  7. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  8. 2015Towards AI-Complete Question Answering: A Set of Prerequisite Toy Taskscited by 309
  9. 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 299
  10. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  11. 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 248
  12. 2021How Many Data Points is a Prompt Worth?cited by 197
  13. 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
  14. 2019Certified Robustness to Adversarial Word Substitutionscited by 164
  15. 2021Template-Based Named Entity Recognition Using BARTcited by 153
  16. 2021Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collectionscited by 128
  17. 2021A Recipe For Arbitrary Text Style Transfer with Large Language Modelscited by 107
  18. 2020The Turking Test: Can Language Models Understand Instructions?cited by 82
  19. 2020Learning from Task Descriptionscited by 78
  20. 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
  21. 2021NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentationcited by 72
  22. 2021Tailor: Generating and Perturbing Text with Semantic Controlscited by 67
  23. 2022InstructionNER: A Multi-Task Instruction-Based Generative Framework for Few-shot NERcited by 31
  24. 2017Measuring the Similarity of Sentential Arguments in Dialogcited by 23
  25. 2020DQI: Measuring Data Quality in NLPcited by 23
  26. 2022Less is More: Summary of Long Instructions is Better for Program Synthesiscited by 17
  27. 2020DQI: A Guide to Benchmark Evaluationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 5; 2024: 1; 2025: 2; 2026: 1.

16 papers.

Sort:

By area:

  1. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  3. 2024Instructional Fingerprinting of Large Language Modelscited by 64
  4. 2023Maybe Only 0.5% Data is Needed: A Preliminary Exploration of Low Training Data Instruction Tuningcited by 56
  5. 2022Is a Question Decomposition Unit All We Need?cited by 37
  6. 2023InstructABSA: Instruction Learning for Aspect Based Sentiment Analysiscited by 22
  7. 2023Instruction Tuned Models are Quick Learnerscited by 21
  8. 2022HELP ME THINK: A Simple Prompting Strategy for Non-experts to Create Customized Content with Modelscited by 17
  9. 2022In-BoXBART: Get Instructions into Biomedical Multi-Task Learningcited by 16
  10. 2023InstructExcel: A Benchmark for Natural Language Instruction in Excelcited by 14
  11. 2022BioTABQA: Instruction Learning for Biomedical Table Question Answeringcited by 13
  12. 2022Lila: A Unified Benchmark for Mathematical Reasoningcited by 13
4 more
  1. 2025Data Mixing Optimization for Supervised Fine-Tuning of Large Language Modelscited by 12
  2. 2026Mid-Training with Self-Generated Data Improves Reinforcement Learning in Language Modelscited by 1
  3. 2023LINGO : Visually Debiasing Natural Language Instructions to Support Task Diversitycited by 0
  4. 2025MetaLint: Easy-to-Hard Generalization for Code Lintingcited by 0