mlsanityData to 7 Oct 2026

In-context Example Selection with Influences

arXiv 2302.11042 ↗First posted 21 Feb 2023cs.CL cs.LGcited by 60

Not on the map. Why?

Abstract (arXiv)

In-context learning (ICL) is a powerful paradigm emerged from large language models (LLMs). Despite its promises, ICL performance is known to be highly sensitive to input examples. In this work, we use textitin-context influences to analyze few-shot ICL performance directly from the in-context examples. Our proposed influence-based example selection method can identify both positive and negative examples, outperforming several baselines when evaluated on 9 SuperGLUE tasks. Our analysis uncovers up to a 16.3% performance gap between using the most negative in-context examples compared to the most positive. In a case study, we apply our influence-based framework to quantify the phenomena of recency bias in example ordering for few-shot ICL.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). Builds on 36 of its 39 references that are in the map. First posted on arXiv 21 Feb 2023.

Builds on

36 of 39 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  9. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  10. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  11. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  12. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
24 more
  1. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  2. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  3. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  4. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  5. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  6. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  7. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  8. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  9. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  10. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 617
  11. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  12. 2021True Few-Shot Learning with Language Modelscited by 343
  13. 2023Faithful Chain-of-Thought Reasoningcited by 285
  14. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 256
  15. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 242
  16. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 215
  17. 2022Datamodels: Predicting Predictions from Training Datacited by 186
  18. 2021Noisy Channel Language Model Prompting for Few-Shot Text Classificationcited by 163
  19. 2019On the Accuracy of Influence Functions for Measuring Group Effectscited by 146
  20. 2020Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functionscited by 144
  21. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
  22. 2022On the Relation between Sensitivity and Accuracy in In-context Learningcited by 71
  23. 2021Reordering Examples Helps during Priming-based Few-Shot Learningcited by 50
  24. 2022TEMPERA: Test-Time Prompting via Reinforcement Learningcited by 50

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 26; 2025: 17; 2026: 8.

60 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  2. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
  3. 2024ContextCite: Attributing Model Generation to Contextcited by 93
  4. 2024Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvementscited by 82
  5. 2023In-Context Learning with Iterative Demonstration Selectioncited by 74
  6. 2023Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regressioncited by 47
  7. 2024Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learningcited by 47
  8. 2023Skill-Based Few-Shot Selection for In-Context Learningcited by 46
  9. 2023Hijacking Large Language Models via Adversarial In-Context Learningcited by 41
  10. 2024Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplarscited by 37
  11. 2023Which Examples to Annotate for In-Context Learning? Towards Effective and Efficient Selectioncited by 36
  12. 2023Large Language Model Instruction Following: A Survey of Progresses and Challengescited by 31
38 more
  1. 2024Compromising Embodied Agents with Contextual Backdoor Attackscited by 29
  2. 2024Teach Better or Show Smarter? On Instructions and Exemplars in Automatic Prompt Optimizationcited by 28
  3. 2024C-ICL: Contrastive In-context Learning for Information Extractioncited by 22
  4. 2024Permissive Information-Flow Analysis for Large Language Modelscited by 22
  5. 2024VipAct: Visual-Perception Enhancement via Specialized VLM Agent Collaboration and Tool-usecited by 17
  6. 2023A Survey on Stability of Learning with Limited Labelled Data and its Sensitivity to the Effects of Randomnesscited by 14
  7. 2024Misconfidence-based Demonstration Selection for LLM In-Context Learningcited by 14
  8. 2024In-Context Example Ordering Guided by Label Distributionscited by 12
  9. 2024Addressing Order Sensitivity of In-Context Demonstration Examples in Causal Language Modelscited by 12
  10. 2025Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex Reasoningcited by 12
  11. 2024In-Context Learning Demonstration Selection via Influence Analysiscited by 11
  12. 2024NICE: To Optimize In-Context Examples or Not?cited by 9
  13. 2024Se2: Sequential Example Selection for In-Context Learningcited by 8
  14. 2024Bayesian Example Selection Improves In-Context Learning for Speech, Text, and Visual Modalitiescited by 7
  15. 2024What Makes In-context Learning Effective for Mathematical Reasoning: A Theoretical Analysiscited by 7
  16. 2025Meta-Prompt Optimization for LLM-Based Sequential Decision Makingcited by 7
  17. 2023Lever LM: Configuring In-Context Sequence to Lever Large Vision Language Modelscited by 6
  18. 2025Add-One-In: Incremental Sample Selection for Large Language Models via a Choice-Based Greedy Paradigmcited by 6
  19. 2025Relic: Enhancing Reward Model Generalization for Low-Resource Indic Languages with Few-Shot Examplescited by 6
  20. 2024LIVE: Learnable In-Context Vector for Visual Question Answeringcited by 5
  21. 2024Data Debiasing with Datamodels (D3M): Improving Subgroup Robustness via Data Selectioncited by 5
  22. 2024PromptRefine: Enhancing Few-Shot Performance on Low-Resource Indic Languages with Example Selection from Related Example Bankscited by 5
  23. 2025OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Modelscited by 5
  24. 2025DICE: Dynamic In-Context Example Selection in LLM Agents via Efficient Knowledge Transfercited by 5
  25. 2024Automatic Combination of Sample Selection Strategies for Few-Shot Learningcited by 4
  26. 2023Challenging Bug Prediction and Repair Models with Synthetic Bugscited by 3
  27. 2024DemoShapley: Valuation of Demonstrations for In-Context Learningcited by 3
  28. 2025Unraveling Indirect In-Context Learning Using Influence Functionscited by 3
  29. 2025Understanding the Generalization of In-Context Learning in Transformers: An Empirical Studycited by 3
  30. 2024GraphIC: A Graph-Based In-Context Example Retrieval Model for Multi-Step Reasoningcited by 2
  31. 2025Monte Carlo Sampling for Analyzing In-Context Examplescited by 2
  32. 2026From Historical Patches to Repair Plans: Outcome-Conditioned Reasoning for Repository-Level Program Repaircited by 2
  33. 2026GraphWalker: Patient Analogy Meets Information Gain for Clinical Reasoning with Large Language Modelscited by 2
  34. 2024On Sensitivity of Learning with Limited Labelled Data to the Effects of Randomness: Impact of Interactions and Systematic Choicescited by 1
  35. 2024Exploring Efficient Foundational Multi-modal Models for Video Summarizationcited by 1
  36. 2025Foundation of Intelligence: Review of Math Word Problems from Human Cognition Perspectivecited by 1
  37. 2025REFLEX: Self-Refining Explainable Fact-Checking via Verdict-Anchored Style Controlcited by 1
  38. 2026Process In-Context Learning: Enhancing Mathematical Reasoning via Dynamic Demonstration Insertioncited by 1

50 most cited of 60 All 60 (JSON)