mlsanityData to 7 Oct 2026

Complementary Explanations for Effective In-Context Learning

arXiv 2211.13892 ↗First posted 25 Nov 2022cs.CLcited by 71

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have exhibited remarkable capabilities in learning from explanations in prompts, but there has been limited understanding of exactly how these explanations function or why they are effective. This work aims to better understand the mechanisms by which explanations are used for in-context learning. We first study the impact of two different factors on the performance of prompts with explanations: the computation trace (the way the solution is decomposed) and the natural language used to express the prompt. By perturbing explanations on three controlled tasks, we show that both factors contribute to the effectiveness of explanations. We further study how to form maximally effective sets of explanations for solving a given test query. We find that LLMs can benefit from the complementarity of the explanation set: diverse reasoning skills shown by different exemplars can lead to better performance. Therefore, we propose a maximal marginal relevance-based exemplar selection approach for constructing exemplar sets that are both relevant as well as complementary, which successfully improves the in-context learning performance across three real-world tasks on multiple LLMs.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Builds on 35 of its 41 references that are in the map. First posted on arXiv 25 Nov 2022.

Builds on

35 of 41 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  12. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
23 more
  1. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  2. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  3. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  4. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  5. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  6. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  7. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  8. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  9. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  10. 2021MetaICL: Learning to Learn In Contextcited by 450
  11. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 288
  12. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 256
  13. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 242
  14. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoningcited by 169
  15. 2021Noisy Channel Language Model Prompting for Few-Shot Text Classificationcited by 163
  16. 2021Meta-learning via Language Model In-context Tuningcited by 163
  17. 2021Constrained Language Models Yield Few-Shot Semantic Parserscited by 162
  18. 2022Language Models of Code are Few-Shot Commonsense Learnerscited by 160
  19. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  20. 2022Rationale-Augmented Ensembles in Language Modelscited by 111
  21. 2022Teaching Algorithmic Reasoning via In-context Learningcited by 111
  22. 2022Prototypical Calibration for Few-shot Learning of Language Modelscited by 58
  23. 2022Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsingcited by 50

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 30; 2024: 21; 2025: 15; 2026: 2.

71 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  3. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 221
  4. 2023Compositional Exemplars for In-context Learningcited by 143
  5. 2022Diverse Demonstrations Improve In-context Compositional Generalizationcited by 136
  6. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  7. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  8. 2023Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examplescited by 84
  9. 2023In-Context Learning with Iterative Demonstration Selectioncited by 74
  10. 2024LLM-based NLG Evaluation: Current Status and Challengescited by 71
  11. 2023Towards Better Chain-of-Thought Prompting Strategies: A Surveycited by 62
  12. 2023Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Modelscited by 61
38 more
  1. 2023Natural Language Reasoning, A Surveycited by 61
  2. 2022Data Curation Alone Can Stabilize In-context Learningcited by 58
  3. 2023Extractive Summarization via ChatGPT for Faithful Summary Generationcited by 58
  4. 2023Coverage-based Example Selection for In-Context Learningcited by 58
  5. 2023Active Learning Principles for In-Context Learning with Large Language Modelscited by 53
  6. 2023Skill-Based Few-Shot Selection for In-Context Learningcited by 46
  7. 2023Towards Understanding In-Context Learning with Contrastive Demonstrations and Saliency Mapscited by 35
  8. 2023PALM: Predicting Actions through Language Modelscited by 28
  9. 2024How Does Code Pretraining Affect Language Model Task Performance?cited by 28
  10. 2023Pushing the Limits of ChatGPT on NLP Taskscited by 27
  11. 2025Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarkscited by 26
  12. 2025Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Modelscited by 25
  13. 2023LLMeBench: A Flexible Framework for Accelerating LLMs Benchmarkingcited by 20
  14. 2023Zero- and Few-Shot Prompting with LLMs: A Comparative Study with Fine-tuned Models for Bangla Sentiment Analysiscited by 20
  15. 2023LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Promptingcited by 17
  16. 2023LAraBench: Benchmarking Arabic AI with Large Language Modelscited by 16
  17. 2024Preemptive Answer "Attacks" on Chain-of-Thought Reasoningcited by 16
  18. 2023Palm: Predicting Actions through Language Models @ Ego4D Long-Term Action Anticipation Challenge 2023cited by 15
  19. 2023SummQA at MEDIQA-Chat 2023:In-Context Learning with GPT-4 for Medical Summarizationcited by 13
  20. 2024A Survey to Recent Progress Towards Understanding In-Context Learningcited by 12
  21. 2024ICLEval: Evaluating In-Context Learning Ability of Large Language Modelscited by 12
  22. 2025Sample Efficient Demonstration Selection for In-Context Learningcited by 12
  23. 2023Diverse Retrieval-Augmented In-Context Learning for Dialogue State Trackingcited by 10
  24. 2024Se2: Sequential Example Selection for In-Context Learningcited by 8
  25. 2024An Empirical Categorization of Prompting Techniques for Large Language Models: A Practitioner's Guidecited by 7
  26. 2024SLM as Guardian: Pioneering AI Safety with Small Language Modelscited by 7
  27. 2024STRUX: An LLM for Decision-Making with Structured Explanationscited by 7
  28. 2023Generating medically-accurate summaries of patient-provider dialogue: A multi-stage approach using large language modelscited by 6
  29. 2023GistScore: Learning Better Representations for In-Context Example Selection with Gist Bottleneckscited by 6
  30. 2024Conversational Complexity for Assessing Risk in Large Language Modelscited by 6
  31. 2025The Role of Diversity in In-Context Learning for Large Language Modelscited by 6
  32. 2025Enhancing Cross-task Transfer of Large Language Models via Activation Steeringcited by 6
  33. 2023Demonstrations Are All You Need: Advancing Offensive Content Paraphrasing using In-Context Learningcited by 5
  34. 2023ICL Markup: Structuring In-Context Learning using Soft-Token Tagscited by 5
  35. 2023Beyond Output Matching: Bidirectional Alignment for Enhanced In-Context Learningcited by 5
  36. 2025Automated Query-Product Relevance Labeling using Large Language Models for E-commerce Searchcited by 5
  37. 2023KwaiYiiMath: Technical Reportcited by 4
  38. 2023Rescue: Ranking LLM Responses with Partial Ordering to Improve Response Generationcited by 4

50 most cited of 71 All 71 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026When Do Explanations Help In-Context Learning? A Comparative Study of Natural Language Explanation Types and Faithfulness
  2. 2024The Power of Adaptation: Boosting In-Context Learning through Adaptive Prompting
  3. 2023In-Context Explainers: Harnessing LLMs for Explaining Black Box Models
  4. 2023OPT-R: Exploring the Role of Explanations in Finetuning and Prompting for Reasoning Skills of Large Language Models
  5. 2022Explanations from Large Language Models Make Small Reasoners Better
  6. 2024Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning
  7. 2025Rethinking the Unsolvable: When In-Context Search Meets Test-Time Scaling
  8. 2024Demonstration Notebook: Finding the Most Suited In-Context Learning Example from Interactions