mlsanityData to 7 Oct 2026

One size doesn't fit all: Predicting the Number of Examples for In-Context Learning

arXiv 2403.06402 ↗First posted 11 Mar 2024cs.CL cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

In-context learning (ICL) refers to the process of adding a small number of localized examples from a training set of labelled data to an LLM's prompt with an objective to effectively control the generative process seeking to improve the downstream task performance. Existing ICL approaches use an identical number of examples (a pre-configured hyper-parameter) for each data instance. Our work alleviates the limitations of this 'one fits all' approach by dynamically predicting the number of examples for each data instance to be used in few-shot inference with LLMs. In particular, we employ a multi-label classifier, the parameters of which are fitted using a training set, where the label for each instance in this training set indicates if using a specific value of k (number of most similar examples from 0 up to a maximum value) leads to correct k-shot downstream predictions. Our experiments on a number of text classification benchmarks show that AICL substantially outperforms standard ICL by up to 17%.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 35 of its 49 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

35 of 49 references.

References · 35

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  9. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  10. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  11. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  12. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
23 more
  1. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  2. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  3. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  4. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  5. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  6. 2022Active Example Selection for In-Context Learningcited by 195
  7. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  8. 2023Active Prompting with Chain-of-Thought for Large Language Modelscited by 154
  9. 2022Diverse Demonstrations Improve In-context Compositional Generalizationcited by 136
  10. 2020Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classificationcited by 98
  11. 2023Few-shot In-context Learning for Knowledge Base Question Answeringcited by 74
  12. 2023Dr.ICL: Demonstration-Retrieved In-context Learningcited by 71
  13. 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 69
  14. 2023Fairness-guided Few-shot Prompting for Large Language Modelscited by 53
  15. 2021Reordering Examples Helps during Priming-based Few-Shot Learningcited by 50
  16. 2023How Does Generative Retrieval Scale to Millions of Passages?cited by 47
  17. 2023Understanding In-Context Learning via Supportive Pretraining Datacited by 41
  18. 2021Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learningcited by 35
  19. 2023In-Context Learning for Text Classification with Many Labelscited by 35
  20. 2023Large Language Model Augmented Narrative Driven Recommendationscited by 31
  21. 2022Deep-QPP: A Pairwise Interaction-based Deep Learning Model for Supervised Query Performance Predictioncited by 13
  22. 2020Choppy: Cut Transformer For Ranked List Truncationcited by 11
  23. 2023Query-specific Variable Depth Pooling via Query Performance Prediction towards Reducing Relevance Assessment Effortcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025OptiSeq: Ordering Examples On-The-Fly for In-Context Learningcited by 6
  2. 2026Predicting Retrieval Utility and Answer Quality in Retrieval-Augmented Generationcited by 2
  3. 2026LURE-RAG: Lightweight Utility-driven Reranking for Efficient RAGcited by 2
  4. 2025HF-RAG: Hierarchical Fusion-based RAG with Multiple Sources and Rankerscited by 1