mlsanityData to 13 Oct 2026

Reframing Instructional Prompts to GPTk's Language

arXiv 2109.07830 ↗First posted 16 Sep 2021cs.CL cs.AI cs.LGcited by 176

Not on the map. Why?

Cited by 21 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

What kinds of instructional prompts are easier to follow for Language Models (LMs)? We study this question by conducting extensive empirical analysis that shed light on important features of successful instructional prompts. Specifically, we study several classes of reframing techniques for manual reformulation of prompts into more effective ones. Some examples include decomposing a complex task instruction into multiple simpler tasks or itemizing instructions into sequential steps. Our experiments compare the zero-shot and few-shot performance of LMs prompted with reframed instructions on 12 NLP tasks across 6 categories. Compared with original instructions, our reframed instructions lead to significant improvements across LMs with different sizes. For example, the same reframed prompts boost few-shot performance of GPT3-series and GPT2-series by 12.5% and 6.7% respectively averaged over all tasks. Furthermore, reframed instructions reduce the number of examples required to prompt LMs in the few-shot setting. We hope these empirically-driven techniques will pave the way towards more effective future prompting algorithms.

Key facts. Cited by 176 papers in the map (data through 13 Oct 2026). Builds on 27 of its 44 references that are in the map. First posted on arXiv 16 Sep 2021.

Builds on

27 of 44 references.

References · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,320
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,884
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,785
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,765
  5. 2019Language Models as Knowledge Bases?cited by 1,931
  6. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,376
  7. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,126
  8. 2019How Can We Know What Language Models Know?cited by 958
  9. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 935
  10. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 890
  11. 2021GPT Understands, Toocited by 716
  12. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 697
15 more
  1. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 605
  2. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 282
  3. 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 250
  4. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 247
  5. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  6. 2019WinoGrande: An Adversarial Winograd Schema Challenge at Scalecited by 201
  7. 2021How Many Data Points is a Prompt Worth?cited by 197
  8. 2021Template-Based Named Entity Recognition Using BARTcited by 153
  9. 2021Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Modelscited by 139
  10. 2019"Going on a vacation" takes longer than "Going for a walk": A Study of Temporal Commonsense Understandingcited by 122
  11. 2019Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoningcited by 100
  12. 2021Improving and Simplifying Pattern Exploiting Trainingcited by 97
  13. 2020Few-Shot Text Generation with Pattern-Exploiting Trainingcited by 87
  14. 2020The Turking Test: Can Language Models Understand Instructions?cited by 82
  15. 2021Reordering Examples Helps during Priming-based Few-Shot Learningcited by 50

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 5; 2022: 41; 2023: 63; 2024: 35; 2025: 20; 2026: 12.

176 papers.

Sort:

By area:

  1. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,896
  2. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,424
  3. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,063
  4. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 865
  5. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 706
  6. 2021MetaICL: Learning to Learn In Contextcited by 452
  7. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 363
  8. 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 305
  9. 2022Can language models learn from explanations in context?cited by 249
  10. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 232
  11. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 216
  12. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 210
38 more
  1. 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 206
  2. 2022Active Example Selection for In-Context Learningcited by 195
  3. 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 189
  4. 2022Language models show human-like content effects on reasoning taskscited by 172
  5. 2023Take a Step Back: Evoking Reasoning via Abstraction in Large Language Modelscited by 158
  6. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  7. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  8. 2022Text and Patterns: For Effective Chain of Thought, It Takes Two to Tangocited by 128
  9. 2022GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Modelscited by 127
  10. 2023AutoMix: Automatically Mixing Language Modelscited by 120
  11. 2023Are Emergent Abilities in Large Language Models just In-Context Learning?cited by 109
  12. 2024Self-Discover: Large Language Models Self-Compose Reasoning Structurescited by 109
  13. 2023Learning Performance-Improving Code Editscited by 105
  14. 2023Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Modelscited by 100
  15. 2023Mitigating Label Biases for In-context Learningcited by 88
  16. 2021MAGMA -- Multimodal Augmentation of Generative Models through Adapter-based Finetuningcited by 85
  17. 2023How FaR Are Large Language Models From Agents with Theory-of-Mind?cited by 76
  18. 2021On Transferability of Prompt Tuning for Natural Language Processingcited by 75
  19. 2022Iteratively Prompt Pre-trained Language Models for Chain of Thoughtcited by 75
  20. 2023Questioning the Survey Responses of Large Language Modelscited by 73
  21. 2023GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learningcited by 70
  22. 2023Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learningcited by 69
  23. 2024LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Modelscited by 68
  24. 2022ProGen: Progressive Zero-shot Dataset Generation via In-context Feedbackcited by 67
  25. 2021Few-Shot Bot: Prompt-Based Learning for Dialogue Systemscited by 64
  26. 2021Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Promptscited by 64
  27. 2022Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scalecited by 64
  28. 2023The Next Chapter: A Study of Large Language Models in Storytellingcited by 58
  29. 2023The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Modelscited by 55
  30. 2022Describing Differences between Text Distributions with Natural Languagecited by 54
  31. 2023A Practical Survey on Zero-shot Prompt Design for In-context Learningcited by 49
  32. 2023Distilling Script Knowledge from Large Language Models for Constrained Language Planningcited by 45
  33. 2023VIP5: Towards Multimodal Foundation Models for Recommendationcited by 45
  34. 2023"According to ...": Prompting Language Models Improves Quoting from Pre-Training Datacited by 42
  35. 2022PERFECT: Prompt-free and Efficient Few-shot Learning with Language Modelscited by 40
  36. 2023Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inferencecited by 40
  37. 2023Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learningcited by 40
  38. 2022SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Modelscited by 38

50 most cited of 176 All 176 (JSON)