# On the Worst Prompt Performance of Large Language Models

arXiv: https://arxiv.org/abs/2406.10248 · HTML: https://mlsanity.com/paper/2406.10248/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 33 references that are in the map.
- First posted on arXiv 8 Jun 2024.

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models](https://mlsanity.com/paper/2306.11698/) (2023)
- [Detecting Pretraining Data from Large Language Models](https://mlsanity.com/paper/2310.16789/) (2023)
- [Learning How to Ask: Querying LMs with Mixtures of Soft Prompts](https://mlsanity.com/paper/2104.06599/) (2021)
- [State of What Art? A Call for Multi-Prompt LLM Evaluation](https://mlsanity.com/paper/2401.00595/) (2023)
- [tinyBenchmarks: evaluating LLMs with fewer examples](https://mlsanity.com/paper/2402.14992/) (2024)
- [PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts](https://mlsanity.com/paper/2306.04528/) (2023)
- [Demystifying Prompts in Language Models via Perplexity Estimation](https://mlsanity.com/paper/2212.04037/) (2022)
- [On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2302.12095/) (2023)
- [GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models](https://mlsanity.com/paper/2203.07281/) (2022)
- [The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance](https://mlsanity.com/paper/2401.03729/) (2024)
- [Prompt Consistency for Zero-Shot Task Generalization](https://mlsanity.com/paper/2205.00049/) (2022)
- [Evaluating the Zero-shot Robustness of Instruction-tuned Language Models](https://mlsanity.com/paper/2306.11270/) (2023)
- [Toward Human Readable Prompt Tuning: Kubrick's The Shining is a good movie, and a good prompt too?](https://mlsanity.com/paper/2212.10539/) (2022)
- [Robustness of Learning from Task Instructions](https://mlsanity.com/paper/2212.03813/) (2022)
- [Exploring Lottery Prompts for Pre-trained Language Models](https://mlsanity.com/paper/2305.19500/) (2023)

## Built on by (most cited first, 30 of 38)

- [Natural-Language Agent Harnesses](https://mlsanity.com/paper/2603.25723/) (2026)
- [ResearchTown: Simulator of Human Research Community](https://mlsanity.com/paper/2412.17767/) (2024)
- [What Prompts Don't Say: Understanding and Managing Underspecification in LLM Prompts](https://mlsanity.com/paper/2505.13360/) (2025)
- [A Survey on the Honesty of Large Language Models](https://mlsanity.com/paper/2409.18786/) (2024)
- [Brittlebench: Quantifying LLM robustness via prompt sensitivity](https://mlsanity.com/paper/2603.13285/) (2026)
- [EvolveRouter: Co-Evolving Routing and Prompt for Multi-Agent Question Answering](https://mlsanity.com/paper/2604.05149/) (2026)
- [SCAM: A Real-World Typographic Robustness Evaluation for Multimodal Foundation Models](https://mlsanity.com/paper/2504.04893/) (2025)
- [Out of Style: RAG's Fragility to Linguistic Variation](https://mlsanity.com/paper/2504.08231/) (2025)
- [More or Less Wrong: A Benchmark for Directional Bias in LLM Comparative Reasoning](https://mlsanity.com/paper/2506.03923/) (2025)
- [Knowledge Integration for Physics-informed Symbolic Regression Using Pre-trained Large Language Models](https://mlsanity.com/paper/2509.03036/) (2025)
- [Learning the Wrong Lessons: Syntactic-Domain Spurious Correlations in Language Models](https://mlsanity.com/paper/2509.21155/) (2025)
- [Automated Prompt Engineering for Cost-Effective Code Generation Using Evolutionary Algorithm](https://mlsanity.com/paper/2408.11198/) (2024)
- [Same Question, Different Words: A Latent Adversarial Framework for Prompt Robustness](https://mlsanity.com/paper/2503.01345/) (2025)
- [Compared to What? Baselines and Metrics for Counterfactual Prompting](https://mlsanity.com/paper/2605.01048/) (2026)
- [Beyond One-Size-Fits-All: Inversion Learning for Highly Effective NLG Evaluation Prompts](https://mlsanity.com/paper/2504.21117/) (2025)
- [ISA-Bench: Benchmarking Instruction Sensitivity for Large Audio Language Models](https://mlsanity.com/paper/2510.23558/) (2025)
- [Within-Model vs Between-Prompt Variability in Large Language Models for Creative Tasks](https://mlsanity.com/paper/2601.21339/) (2026)
- [Investigating LLM Variability in Personalized Conversational Information Retrieval](https://mlsanity.com/paper/2510.03795/) (2025)
- [LPDS: Evaluating LLM Robustness Through Logic-Preserving Difficulty Scaling](https://mlsanity.com/paper/2605.15393/) (2026)
- [Same Patient, Different Words, Different Diagnosis? Evaluating Semantic Stability in Clinical LLMs](https://mlsanity.com/paper/2605.30646/) (2026)
- [When Guardrails Look Effective: Construct Validity Failures in LLM Agent Commerce Evaluation](https://mlsanity.com/paper/2609.01519/) (2026)
- [Re-Evaluating Code LLM Benchmarks Under Semantic Mutation](https://mlsanity.com/paper/2506.17369/) (2025)
- [Foundational Design Principles and Patterns for Building Robust and Adaptive GenAI-Native Systems](https://mlsanity.com/paper/2508.15411/) (2025)
- [Type-Compliant Adaptation Cascades: Adapting Programmatic LM Workflows to Data](https://mlsanity.com/paper/2508.18244/) (2025)
- [TATRA: Training-Free Instance-Adaptive Prompting Through Rephrasing and Aggregation](https://mlsanity.com/paper/2603.03298/) (2026)
- [Shaping Schema via Language Representation as the Next Frontier for LLM Intelligence Expanding](https://mlsanity.com/paper/2605.09271/) (2026)
- [Prompt-Model Interaction Reaches the Fixed Points: A deterministic, task-free structural readout -- and the factorizations of it that failed](https://mlsanity.com/paper/2608.21315/) (2026)
- [Developing A Framework to Support Human Evaluation of Bias in Generated Free Response Text](https://mlsanity.com/paper/2505.03053/) (2025)
- [From Abstract to Contextual: What LLMs Still Cannot Do in Mathematics](https://mlsanity.com/paper/2601.23048/) (2026)
- [Disentangling Deception and Hallucination Failures in LLMs](https://mlsanity.com/paper/2602.14529/) (2026)

## Abstract (arXiv)

The performance of large language models (LLMs) is acutely sensitive to the phrasing of prompts, which raises significant concerns about their reliability in real-world scenarios. Existing studies often divide prompts into task-level instructions and case-level inputs and primarily focus on evaluating and improving robustness against variations in tasks-level instructions. However, this setup fails to fully address the diversity of real-world user queries and assumes the existence of task-specific datasets. To address these limitations, we introduce RobustAlpacaEval, a new benchmark that consists of semantically equivalent case-level queries and emphasizes the importance of using the worst prompt performance to gauge the lower bound of model performance. Extensive experiments on RobustAlpacaEval with ChatGPT and six open-source LLMs from the Llama, Mistral, and Gemma families uncover substantial variability in model performance; for instance, a difference of 45.48% between the worst and best performance for the Llama-2-70B-chat model, with its worst performance dipping as low as 9.38%. We further illustrate the difficulty in identifying the worst prompt from both model-agnostic and model-dependent perspectives, emphasizing the absence of a shortcut to characterize the worst prompt. We also attempt to enhance the worst prompt performance using existing prompt engineering and prompt consistency methods, but find that their impact is limited. These findings underscore the need to create more resilient LLMs that can maintain high performance across diverse prompts. Data and code are available at https://github.com/cbwbuaa/On-the-Worst-Prompt- Performance-of-LLMs.