# Efficient multi-prompt evaluation of LLMs

arXiv: https://arxiv.org/abs/2405.17202 · HTML: https://mlsanity.com/paper/2405.17202/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 54 references that are in the map.
- First posted on arXiv 27 May 2024.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://mlsanity.com/paper/2204.07705/) (2022)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [State of What Art? A Call for Multi-Prompt LLM Evaluation](https://mlsanity.com/paper/2401.00595/) (2023)
- [tinyBenchmarks: evaluating LLMs with fewer examples](https://mlsanity.com/paper/2402.14992/) (2024)
- [GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models](https://mlsanity.com/paper/2203.07281/) (2022)
- [Anchor Points: Benchmarking Models with Much Fewer Examples](https://mlsanity.com/paper/2309.08638/) (2023)
- [Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvements](https://mlsanity.com/paper/2401.06766/) (2024)
- [Efficient Benchmarking of Language Models](https://mlsanity.com/paper/2308.11696/) (2023)
- [Comparing Test Sets with Item Response Theory](https://mlsanity.com/paper/2106.00840/) (2021)
- [Prompt Engineering a Prompt Engineer](https://mlsanity.com/paper/2311.05661/) (2023)
- [PromptBench: A Unified Library for Evaluation of Large Language Models](https://mlsanity.com/paper/2312.07910/) (2023)
- [How Predictable Are Large Language Model Capabilities? A Case Study on BIG-bench](https://mlsanity.com/paper/2305.14947/) (2023)
- [Mind the instructions: a holistic evaluation of consistency and interactions in prompt-based learning](https://mlsanity.com/paper/2310.13486/) (2023)
- [Fixed-Budget Best-Arm Identification in Structured Bandits](https://mlsanity.com/paper/2106.04763/) (2021)
- [LMentry: A Language Model Benchmark of Elementary Language Tasks](https://mlsanity.com/paper/2211.02069/) (2022)
- [Label-Efficient Model Selection for Text Generation](https://mlsanity.com/paper/2402.07891/) (2024)
- [SPELL: Semantic Prompt Evolution based on a LLM](https://mlsanity.com/paper/2310.01260/) (2023)
- [The ICL Consistency Test](https://mlsanity.com/paper/2312.04945/) (2023)

## Built on by (most cited first, 30 of 73)

- [POSIX: A Prompt Sensitivity Index For Large Language Models](https://mlsanity.com/paper/2410.02185/) (2024)
- [Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs](https://mlsanity.com/paper/2509.01790/) (2025)
- [How Benchmark Prediction from Fewer Data Misses the Mark](https://mlsanity.com/paper/2506.07673/) (2025)
- [Sloth: scaling laws for LLM skills to predict multi-benchmark performance across families](https://mlsanity.com/paper/2412.06540/) (2024)
- [Noisy but Valid: Robust Statistical Evaluation of LLMs with Imperfect Judges](https://mlsanity.com/paper/2601.20913/) (2026)
- [DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation](https://mlsanity.com/paper/2503.01622/) (2025)
- [Towards LLMs Robustness to Changes in Prompt Format Styles](https://mlsanity.com/paper/2504.06969/) (2025)
- [SCORE: Systematic COnsistency and Robustness Evaluation for Large Language Models](https://mlsanity.com/paper/2503.00137/) (2025)
- [LiveXiv -- A Multi-Modal Live Benchmark Based on Arxiv Papers Content](https://mlsanity.com/paper/2410.10783/) (2024)
- [Persona-Augmented Benchmarking: Evaluating LLMs Across Diverse Writing Styles](https://mlsanity.com/paper/2507.22168/) (2025)
- [ReliableEval: A Recipe for Stochastic LLM Evaluation via Method of Moments](https://mlsanity.com/paper/2505.22169/) (2025)
- [MedGUIDE: Benchmarking Clinical Decision-Making in Large Language Models](https://mlsanity.com/paper/2505.11613/) (2025)
- [Prompt Stability Matters: Evaluating and Optimizing Auto-Generated Prompt in General-Purpose Systems](https://mlsanity.com/paper/2505.13546/) (2025)
- [Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry](https://mlsanity.com/paper/2601.22588/) (2026)
- [Evalita-LLM: Benchmarking Large Language Models on Italian](https://mlsanity.com/paper/2502.02289/) (2025)
- [Accelerating Unbiased LLM Evaluation via Synthetic Feedback](https://mlsanity.com/paper/2502.10563/) (2025)
- [Towards Reliable LLM Evaluation: Correcting the Winner's Curse in Adaptive Benchmarking](https://mlsanity.com/paper/2605.05973/) (2026)
- [PairBench: Are Vision-Language Models Reliable at Comparing What They See?](https://mlsanity.com/paper/2502.15210/) (2025)
- [Uncertainty-Aware Answer Selection for Improved Reasoning in Multi-LLM Systems](https://mlsanity.com/paper/2510.02377/) (2025)
- [ErrorMap and ErrorAtlas: Charting the Failure Landscape of Large Language Models](https://mlsanity.com/paper/2601.15812/) (2026)
- [Rank Intervals for Leaderboards: A Hierarchical Framework for Model Evaluation](https://mlsanity.com/paper/2606.08679/) (2026)
- [Same Question, Different Words: A Latent Adversarial Framework for Prompt Robustness](https://mlsanity.com/paper/2503.01345/) (2025)
- [The Hidden Structure -- Improving Legal Document Understanding Through Explicit Text Formatting](https://mlsanity.com/paper/2505.12837/) (2025)
- [Beyond One-Size-Fits-All: Inversion Learning for Highly Effective NLG Evaluation Prompts](https://mlsanity.com/paper/2504.21117/) (2025)
- [Finding Answers in Thought Matters: Revisiting Evaluation on Large Language Models with Reasoning](https://mlsanity.com/paper/2510.14773/) (2025)
- [Bilingual Evaluation of Language Models on General Knowledge in University Entrance Exams with Minimal Contamination](https://mlsanity.com/paper/2409.12746/) (2024)
- [Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs](https://mlsanity.com/paper/2510.14242/) (2025)
- [SineProject: Machine Unlearning for Stable Vision Language Alignment](https://mlsanity.com/paper/2511.18444/) (2025)
- [Ensembling Language Models with Sequential Monte Carlo](https://mlsanity.com/paper/2603.05432/) (2026)
- [Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results](https://mlsanity.com/paper/2606.14516/) (2026)

## Abstract (arXiv)

Most popular benchmarks for comparing LLMs rely on a limited set of prompt templates, which may not fully capture the LLMs' abilities and can affect the reproducibility of results on leaderboards. Many recent works empirically verify prompt sensitivity and advocate for changes in LLM evaluation. In this paper, we consider the problem of estimating the performance distribution across many prompt variants instead of finding a single prompt to evaluate with. We introduce PromptEval, a method for estimating performance across a large set of prompts borrowing strength across prompts and examples to produce accurate estimates under practical evaluation budgets. The resulting distribution can be used to obtain performance quantiles to construct various robust performance metrics (e.g., top 95% quantile or median). We prove that PromptEval consistently estimates the performance distribution and demonstrate its efficacy empirically on three prominent LLM benchmarks: MMLU, BIG-bench Hard, and LMentry; for example, PromptEval can accurately estimate performance quantiles across 100 prompt templates on MMLU with a budget equivalent to two single-prompt evaluations. Moreover, we show how PromptEval can be useful in LLM-as-a-judge and best prompt identification applications.