# Exploring the Role of Diversity in Example Selection for In-Context Learning

arXiv: https://arxiv.org/abs/2505.01842 · HTML: https://mlsanity.com/paper/2505.01842/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 May 2025.

## Built on by (most cited first, 12 of 12)

- [Catch Me If You Can? Not Yet: LLMs Still Struggle to Imitate the Implicit Writing Styles of Everyday Authors](https://mlsanity.com/paper/2509.14543/) (2025)
- [Many-Shot CoT-ICL: Making In-Context Learning Truly Learn](https://mlsanity.com/paper/2605.13511/) (2026)
- [CG-TTRL: Context-Guided Test-Time Reinforcement Learning for On-Device Large Language Models](https://mlsanity.com/paper/2511.06430/) (2025)
- [CIEGAD: Cluster-Conditioned Interpolative and Extrapolative Framework for Geometry-Aware and Domain-Aligned Data Augmentation](https://mlsanity.com/paper/2512.10178/) (2025)
- [Structured Semantic Information Helps Retrieve Better Examples for In-Context Learning Applied to Few-Shot Relation Extraction](https://mlsanity.com/paper/2601.20803/) (2026)
- [Combining Distantly Supervised Models with In Context Learning for Monolingual and Cross-Lingual Relation Extraction](https://mlsanity.com/paper/2510.18344/) (2025)
- [When Does Context Help? Error Dynamics of Contextual Information in Large Language Models](https://mlsanity.com/paper/2602.08294/) (2026)
- [Design-Specification Tiling for ICL-based CAD Code Generation](https://mlsanity.com/paper/2603.12712/) (2026)
- [SCOPE: A Lightweight-training LLM Framework for Air Traffic Control Readback Monitoring](https://mlsanity.com/paper/2605.29543/) (2026)
- [Toward Robust In-Context Learning: Leveraging Out-of-distribution Proxies for Target Inaccessible Demonstration Retrieval](https://mlsanity.com/paper/2606.00014/) (2026)
- [In-Context Learning for Wound Classification with Small Multimodal Language Models](https://mlsanity.com/paper/2607.18819/) (2026)
- [Surprising Effectiveness of Self-Demonstrations in Enhancing Schema-Ontology Mapping with LLMs](https://mlsanity.com/paper/2609.13776/) (2026)

## Abstract (arXiv)

In-Context Learning (ICL) has gained prominence due to its ability to perform tasks without requiring extensive training data and its robustness to noisy labels. A typical ICL workflow involves selecting localized examples relevant to a given input using sparse or dense embedding-based similarity functions. However, relying solely on similarity-based selection may introduce topical biases in the retrieved contexts, potentially leading to suboptimal downstream performance. We posit that reranking the retrieved context to enhance topical diversity can improve downstream task performance. To achieve this, we leverage maximum marginal relevance (MMR) which balances topical similarity with inter-example diversity. Our experimental results demonstrate that diversifying the selected examples leads to consistent improvements in downstream performance across various context sizes and similarity functions. The implementation of our approach is made available at https://github.com/janak11111/Diverse-ICL.