mlsanityData to 7 Oct 2026

Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm

arXiv 2102.07350 ↗First posted 15 Feb 2021cs.CL cs.AIcited by 599

Not on the map. Why?

Cited by 51 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Prevailing methods for mapping large generative language models to supervised tasks may fail to sufficiently probe models' novel capabilities. Using GPT-3 as a case study, we show that 0-shot prompts can significantly outperform few-shot prompts. We suggest that the function of few-shot examples in these cases is better described as locating an already learned task rather than meta-learning. This analysis motivates rethinking the role of prompts in controlling and evaluating powerful language models. In this work, we discuss methods of prompt programming, emphasizing the usefulness of considering prompts through the lens of natural language. We explore techniques for exploiting the capacity of narratives and cultural anchors to encode nuanced intentions and techniques for encouraging deconstruction of a problem into components before producing a verdict. Informed by this more encompassing theory of prompt programming, we also introduce the idea of a metaprompt that seeds the model to generate its own natural language prompts for a range of tasks. Finally, we discuss how these more general methods of interacting with language models can be incorporated into existing and future benchmarks and practical applications.

Key facts. Cited by 599 papers in the map (data through 7 Oct 2026). Builds on 13 of its 26 references that are in the map. First posted on arXiv 15 Feb 2021.

Builds on

13 of 26 references.

References · 13

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  7. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  8. 2018Hierarchical Neural Story Generationcited by 1,182
  9. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  10. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  11. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  12. 2019Multi-step Reasoning via Recurrent Dual Attention for Visual Dialogcited by 41
1 more
  1. 2020Multi-Step Inference for Reasoning Over Paragraphscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 72; 2023: 171; 2024: 138; 2025: 133; 2026: 63.

599 papers.

Sort:

By area:

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2022Emergent Abilities of Large Language Modelscited by 2,483
  6. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  7. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  9. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  10. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  11. 2023Large Language Models as Optimizerscited by 830
  12. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
38 more
  1. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  2. 2021GPT Understands, Toocited by 715
  3. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  4. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  5. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  6. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  7. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  8. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  9. 2022Using cognitive psychology to understand GPT-3cited by 389
  10. 2023Larger language models do in-context learning differentlycited by 386
  11. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 376
  12. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  13. 2023Function Vectors in Large Language Modelscited by 292
  14. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 273
  15. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  16. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 256
  17. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 256
  18. 2022Can language models learn from explanations in context?cited by 248
  19. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 247
  20. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 243
  21. 2023One for All: Towards Training One Graph Model for All Classification Taskscited by 239
  22. 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
  23. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 230
  24. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 222
  25. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
  26. 2022Optimizing Prompts for Text-to-Image Generationcited by 204
  27. 2023An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generationcited by 193
  28. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 185
  29. 2021KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extractioncited by 180
  30. 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
  31. 2022Can Foundation Models Wrangle Your Data?cited by 173
  32. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithfulcited by 166
  33. 2024The pitfalls of next-token predictioncited by 156
  34. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 154
  35. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  36. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  37. 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 144
  38. 2023Auditing large language models: a three-layered approachcited by 135

50 most cited of 599 All 599 (JSON)