# Opt-ICL at LeWiDi-2025: Maximizing In-Context Signal from Rater Examples via Meta-Learning

arXiv: https://arxiv.org/abs/2510.07105 · HTML: https://mlsanity.com/paper/2510.07105/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 8 Oct 2025.

## Builds on (most cited first, 11 of 11)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [MetaICL: Learning to Learn In Context](https://mlsanity.com/paper/2110.15943/) (2021)
- [Prompt Cache: Modular Attention Reuse for Low-Latency Inference](https://mlsanity.com/paper/2311.04934/) (2023)
- [Meta-Learning: A Survey](https://mlsanity.com/paper/1810.03548/) (2018)
- [DICES Dataset: Diversity in Conversational AI Evaluation for Safety](https://mlsanity.com/paper/2306.11247/) (2023)
- [VariErr NLI: Separating Annotation Error from Human Label Variation](https://mlsanity.com/paper/2403.01931/) (2024)
- [Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptions](https://mlsanity.com/paper/2502.20897/) (2025)
- [Value Profiles for Encoding Human Variation](https://mlsanity.com/paper/2503.15484/) (2025)
- [Spectrum Tuning: Post-Training for Distributional Coverage and In-Context Steerability](https://mlsanity.com/paper/2510.06084/) (2025)

## Built on by (most cited first, 1 of 1)

- [LeWiDi-2025 at NLPerspectives: Third Edition of the Learning with Disagreements Shared Task](https://mlsanity.com/paper/2510.08460/) (2025)

## Abstract (arXiv)

Many natural language processing (NLP) tasks involve subjectivity, ambiguity, or legitimate disagreement between annotators. In this paper, we outline our system for modeling human variation. Our system leverages language models' (LLMs) in-context learning abilities, along with a two-step meta-learning training procedure for 1) post-training on many datasets requiring in-context learning and 2) specializing the model via in-context meta-learning to the particular data distribution of interest. We also evaluate the performance of our system submission to the Learning With Disagreements (LeWiDi) competition, where it was the overall winner on both tasks. Additionally, we perform an ablation study to measure the importance of each system component. We find that including rater examples in-context is crucial for our system's performance, dataset-specific fine-tuning is helpful on the larger datasets, post-training on other in-context datasets is helpful on one of the competition datasets, and that performance improves with model scale.