# Bayesian Example Selection Improves In-Context Learning for Speech, Text, and Visual Modalities

arXiv: https://arxiv.org/abs/2404.14716 · HTML: https://mlsanity.com/paper/2404.14716/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 23 Apr 2024.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge](https://mlsanity.com/paper/1906.00067/) (2019)
- [EVA-CLIP: Improved Training Techniques for CLIP at Scale](https://mlsanity.com/paper/2303.15389/) (2023)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [Multimodal Few-Shot Learning with Frozen Language Models](https://mlsanity.com/paper/2106.13884/) (2021)
- [Learning To Retrieve Prompts for In-Context Learning](https://mlsanity.com/paper/2112.08633/) (2021)
- [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022)
- [Generative Multimodal Models are In-Context Learners](https://mlsanity.com/paper/2312.13286/) (2023)
- [Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning](https://mlsanity.com/paper/2209.14610/) (2022)
- [Selective Annotation Makes Language Models Better Few-Shot Learners](https://mlsanity.com/paper/2209.01975/) (2022)
- [Active Example Selection for In-Context Learning](https://mlsanity.com/paper/2211.04486/) (2022)
- [Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning](https://mlsanity.com/paper/2301.11916/) (2023)
- [In-Context Learning with Iterative Demonstration Selection](https://mlsanity.com/paper/2310.09881/) (2023)
- [In-context Example Selection with Influences](https://mlsanity.com/paper/2302.11042/) (2023)
- [Skill-Based Few-Shot Selection for In-Context Learning](https://mlsanity.com/paper/2305.14210/) (2023)
- [Which Examples to Annotate for In-Context Learning? Towards Effective and Efficient Selection](https://mlsanity.com/paper/2310.20046/) (2023)
- [Can Whisper perform speech-based in-context learning?](https://mlsanity.com/paper/2309.07081/) (2023)
- [COSMIC: Data Efficient Instruction-tuning For Speech In-Context Learning](https://mlsanity.com/paper/2311.02248/) (2023)
- [In-Context Demonstration Selection with Cross Entropy Difference](https://mlsanity.com/paper/2305.14726/) (2023)
- [Misconfidence-based Demonstration Selection for LLM In-Context Learning](https://mlsanity.com/paper/2401.06301/) (2024)

## Built on by (most cited first, 8 of 8)

- [VideoICL: Confidence-based Iterative In-context Learning for Out-of-Distribution Video Understanding](https://mlsanity.com/paper/2412.02186/) (2024)
- [BayesAdapter: enhanced uncertainty estimation in CLIP few-shot adaptation](https://mlsanity.com/paper/2412.09718/) (2024)
- [Explaining Mixtures of Sources in News Articles](https://mlsanity.com/paper/2411.05192/) (2024)
- [MetaSICL: Globalizing Auditory LLMs for Underserved Speakers and Languages via Meta Speech In-Context Learning](https://mlsanity.com/paper/2601.18904/) (2026)
- [In-context Pre-trained Time-Series Foundation Models adapt to Unseen Tasks](https://mlsanity.com/paper/2602.20307/) (2026)
- [Exploring Imbalanced Annotations for Effective In-Context Learning](https://mlsanity.com/paper/2502.04037/) (2025)
- [In-Context Adaptation of Encoder-Decoder Models in Speech Recognition](https://mlsanity.com/paper/2609.33865/) (2026)
- [Bayesian Test-Time Inference of Task-Aligned Similarity from Weak Interactive Feedback](https://mlsanity.com/paper/2610.18656/) (2026)

## Abstract (arXiv)

Large language models (LLMs) can adapt to new tasks through in-context learning (ICL) based on a few examples presented in dialogue history without any model parameter update. Despite such convenience, the performance of ICL heavily depends on the quality of the in-context examples presented, which makes the in-context example selection approach a critical choice. This paper proposes a novel Bayesian in-Context example Selection method (ByCS) for ICL. Extending the inference probability conditioned on in-context examples based on Bayes' theorem, ByCS focuses on the inverse inference conditioned on test input. Following the assumption that accurate inverse inference probability (likelihood) will result in accurate inference probability (posterior), in-context examples are selected based on their inverse inference results. Diverse and extensive cross-tasking and cross-modality experiments are performed with speech, text, and image examples. Experimental results show the efficacy and robustness of our ByCS method on various models, tasks and modalities.