mlsanityData to 7 Oct 2026

Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering

arXiv 2212.10375 ↗First posted 20 Dec 2022cs.CL cs.AIcited by 158

Not on the map. Why?

Cited by 20 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Despite the surprising few-shot performance of in-context learning (ICL), it is still a common practice to randomly sample examples to serve as context. This paper advocates a new principle for ICL: self-adaptive in-context learning. The self-adaption mechanism is introduced to help each sample find an in-context example permutation (i.e., selection and ordering) that can derive the correct prediction, thus maximizing performance. To validate the effectiveness of self-adaptive ICL, we propose a general select-then-rank framework and instantiate it with new selection and ranking algorithms. Upon extensive evaluation on eight different NLP datasets, our self-adaptive ICL method achieves a 40% relative improvement over the common practice setting. Further analysis reveals the enormous potential of self-adaptive ICL that it might be able to close the gap between ICL and finetuning given more advanced algorithms. Our code is released to facilitate future research in this area: https://github.com/Shark-NLP/self-adaptive-ICL

Key facts. Cited by 158 papers in the map (data through 7 Oct 2026). Builds on 30 of its 49 references that are in the map. First posted on arXiv 20 Dec 2022.

Builds on

30 of 49 references.

References · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  5. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  6. 2022Emergent Abilities of Large Language Modelscited by 2,483
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  8. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  9. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  10. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  11. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  12. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
18 more
  1. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  2. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  3. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  4. 2019How Can We Know What Language Models Know?cited by 953
  5. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  6. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  7. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  8. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  9. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
  10. 2022Transformer Memory as a Differentiable Search Indexcited by 242
  11. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 242
  12. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
  13. 2022GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Modelscited by 127
  14. 2022Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrationscited by 116
  15. 2022An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labelscited by 88
  16. 2018The Description Length of Deep Learning Modelscited by 71
  17. 2022ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalizationcited by 54
  18. 2021Reordering Examples Helps during Priming-based Few-Shot Learningcited by 50

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 24; 2024: 58; 2025: 56; 2026: 19.

158 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2022A Survey on In-context Learningcited by 717
  3. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
  4. 2024Long-context LLMs Struggle with Long In-context Learningcited by 285
  5. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
  6. 2023Efficient Large Language Models: A Surveycited by 183
  7. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
  8. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  9. 2024MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generationcited by 103
  10. 2024Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvementscited by 82
  11. 2023Boosting Language Models Reasoning with Chain-of-Knowledge Promptingcited by 77
  12. 2024Inference Scaling for Long-Context Retrieval Augmented Generationcited by 65
38 more
  1. 2023INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Modelscited by 62
  2. 2024Agents in Software Engineering: Survey, Landscape, and Visioncited by 60
  3. 2023Coverage-based Example Selection for In-Context Learningcited by 58
  4. 2024Revisiting Demonstration Selection Strategies in In-Context Learningcited by 57
  5. 2024Let's Learn Step by Step: Enhancing In-Context Learning Ability with Curriculum Learningcited by 47
  6. 2023Skill-Based Few-Shot Selection for In-Context Learningcited by 46
  7. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Modelscited by 43
  8. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  9. 2023In-Context Learning with Many Demonstration Examplescited by 39
  10. 2024How Do Nonlinear Transformers Learn and Generalize in In-Context Learning?cited by 35
  11. 2025Emergent Abilities in Large Language Models: A Surveycited by 34
  12. 2024The What, Why, and How of Context Length Extension Techniques in Large Language Models -- A Detailed Surveycited by 32
  13. 2023Large Language Model Instruction Following: A Survey of Progresses and Challengescited by 31
  14. 2024What Factors Affect Multi-Modal In-Context Learning? An In-Depth Explorationcited by 31
  15. 2024Smoothie: Label Free Language Model Routingcited by 31
  16. 2023Instruct Me More! Random Prompting for Visual In-Context Learningcited by 30
  17. 2025TACO: Enhancing Multimodal In-context Learning via Task Mapping-Guided Sequence Configurationcited by 23
  18. 2023Supervised Knowledge Makes Large Language Models Better In-context Learnerscited by 20
  19. 2025U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystackcited by 20
  20. 2023Chain of Thought with Explicit Evidence Reasoning for Few-shot Relation Extractioncited by 19
  21. 2024Order Matters: Exploring Order Sensitivity in Multimodal Large Language Modelscited by 19
  22. 2025HeuriGym: An Agentic Benchmark for LLM-Crafted Heuristics in Combinatorial Optimizationcited by 19
  23. 2025Make LVLMs Focus: Context-Aware Attention Modulation for Better Multimodal In-Context Learningcited by 18
  24. 2024Task Facet Learning: A Structured Approach to Prompt Optimizationcited by 17
  25. 2023BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Modelscited by 16
  26. 2023Iterative Forward Tuning Boosts In-Context Learning in Language Modelscited by 15
  27. 2024On the Noise Robustness of In-Context Learning for Text Generationcited by 15
  28. 2023A Survey on Stability of Learning with Limited Labelled Data and its Sensitivity to the Effects of Randomnesscited by 14
  29. 2024CLEAR: Can Language Models Really Understand Causal Graphs?cited by 14
  30. 2023Improving Probability-based Prompt Selection Through Unified Evaluation and Analysiscited by 13
  31. 2025MMRAG: Multi-Mode Retrieval-Augmented Generation with Large Language Models for Biomedical In-Context Learningcited by 13
  32. 2023Generating Data for Symbolic Language with Large Language Modelscited by 12
  33. 2024In-Context Example Ordering Guided by Label Distributionscited by 12
  34. 2024Addressing Order Sensitivity of In-Context Demonstration Examples in Causal Language Modelscited by 12
  35. 2025Mimic In-Context Learning for Multimodal Taskscited by 12
  36. 2025APT: Improving Specialist LLM Performance with Weakness Case Acquisition and Iterative Preference Trainingcited by 12
  37. 2024In-Context Learning Demonstration Selection via Influence Analysiscited by 11
  38. 2024How Does the Textual Information Affect the Retrieval of Multimodal In-Context Learning?cited by 11

50 most cited of 158 All 158 (JSON)