mlsanityData to 7 Oct 2026

Can language models learn from explanations in context?

arXiv 2204.02329 ↗First posted 5 Apr 2022cs.CL cs.AI cs.LGcited by 248

Not on the map. Why?

Cited by 62 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Language Models (LMs) can perform new tasks by adapting to a few in-context examples. For humans, explanations that connect examples to task principles can improve learning. We therefore investigate whether explanations of few-shot examples can help LMs. We annotate questions from 40 challenging tasks with answer explanations, and various matched control explanations. We evaluate how different types of explanations, instructions, and controls affect zero- and few-shot performance. We analyze these results using statistical multilevel modeling techniques that account for the nested dependencies among conditions, tasks, prompts, and models. We find that explanations can improve performance -- even without tuning. Furthermore, explanations hand-tuned for performance on a small validation set offer substantially larger benefits, and building a prompt by selecting examples and explanations together substantially improves performance over selecting examples alone. Finally, even untuned explanations outperform carefully matched controls, suggesting that the benefits are due to the link between an example and its explanation, rather than lower-level features. However, only large models benefit. In summary, explanations can support the in-context learning of large LMs on challenging tasks.

Key facts. Cited by 248 papers in the map (data through 7 Oct 2026). Builds on 48 of its 78 references that are in the map. First posted on arXiv 5 Apr 2022.

Builds on

48 of 78 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  9. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  10. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  11. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  12. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
36 more
  1. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  2. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  3. 2021Ethical and social risks of harm from Language Modelscited by 957
  4. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  5. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  6. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  7. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
  8. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  9. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  10. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  11. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  12. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  13. 2021True Few-Shot Learning with Language Modelscited by 343
  14. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  15. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 256
  16. 2022Predictability and Surprise in Large Generative Modelscited by 209
  17. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  18. 2021How Many Data Points is a Prompt Worth?cited by 197
  19. 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
  20. 2020Unsupervised Commonsense Question Answering with Self-Talkcited by 161
  21. 2016Generating Visual Explanationscited by 156
  22. 2020WT5?! Training Text-to-Text Models to Explain their Predictionscited by 148
  23. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 141
  24. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
  25. 2021Few-Shot Self-Rationalization with Natural Language Promptscited by 83
  26. 2017Learning with Latent Languagecited by 75
  27. 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
  28. 2020Out of Order: How Important Is The Sequential Order of Words in a Sentence in Natural Language Understanding Tasks?cited by 72
  29. 2021Explanation-Based Human Debugging of NLP Models: A Surveycited by 53
  30. 2021What Context Features Can Transformer Language Models Use?cited by 49
  31. 2021Why AI is Harder Than We Thinkcited by 40
  32. 2021Tell me why! Explanations support learning relational and causal structurecited by 36
  33. 2020Towards Interpretable Natural Language Understanding with Explanations as Latent Variablescited by 30
  34. 2019Shaping Visual Representations with Language for Few-shot Classificationcited by 27
  35. 2021Symbolic Behaviour in Artificial Intelligencecited by 24
  36. 2021Teaching Autoregressive Language Models Complex Tasks By Demonstrationcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 38; 2023: 84; 2024: 64; 2025: 50; 2026: 12.

248 papers.

Sort:

By area:

  1. 2022Emergent Abilities of Large Language Modelscited by 2,483
  2. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  4. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  5. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  6. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  7. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  8. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
  9. 2022Large Language Models Can Self-Improvecited by 656
  10. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  11. 2022Using cognitive psychology to understand GPT-3cited by 389
  12. 2023Explainability for Large Language Models: A Surveycited by 331
38 more
  1. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  2. 2023Reward Design with Language Modelscited by 257
  3. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 256
  4. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  5. 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 244
  6. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 231
  7. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  8. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 175
  9. 2022Language models show human-like content effects on reasoning taskscited by 171
  10. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoningcited by 169
  11. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  12. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  13. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 154
  14. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  15. 2023The Learnability of In-Context Learningcited by 126
  16. 2023Training Language Models with Language Feedback at Scalecited by 125
  17. 2022Explanations from Large Language Models Make Small Reasoners Bettercited by 123
  18. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 122
  19. 2023Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Datacited by 119
  20. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 119
  21. 2024C3oT: Generating Shorter Chain-of-Thought without Compromising Effectivenesscited by 117
  22. 2022Rationale-Augmented Ensembles in Language Modelscited by 111
  23. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 110
  24. 2023Are Emergent Abilities in Large Language Models just In-Context Learning?cited by 109
  25. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 106
  26. 2023Text Classification via Large Language Modelscited by 104
  27. 2023Why think step by step? Reasoning emerges from the locality of experiencecited by 103
  28. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  29. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  30. 2023When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunitiescited by 89
  31. 2023A Theory of Emergent In-Context Learning as Implicit Structure Inductioncited by 86
  32. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 83
  33. 2023Improving Code Generation by Training with Natural Language Feedbackcited by 75
  34. 2023Few-shot In-context Learning for Knowledge Base Question Answeringcited by 74
  35. 2023Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillingscited by 68
  36. 2022Recitation-Augmented Language Modelscited by 65
  37. 2023Leveraging Large Language Models in Conversational Recommender Systemscited by 63
  38. 2022Parsel: Algorithmic Reasoning with Language Models by Composing Decompositionscited by 62

50 most cited of 248 All 248 (JSON)