mlsanityData to 7 Oct 2026

Examining the robustness of LLM evaluation to the distributional assumptions of benchmarks

arXiv 2404.16966 ↗First posted 25 Apr 2024cs.CLcited by 31

Not on the map. Why?

Abstract (arXiv)

Benchmarks have emerged as the central approach for evaluating Large Language Models (LLMs). The research community often relies on a model's average performance across the test prompts of a benchmark to evaluate the model's performance. This is consistent with the assumption that the test prompts within a benchmark represent a random sample from a real-world distribution of interest. We note that this is generally not the case; instead, we hold that the distribution of interest varies according to the specific use case. We find that (1) the correlation in model performance across test prompts is non-random, (2) accounting for correlations across test prompts can change model rankings on major benchmarks, (3) explanatory factors for these correlations include semantic similarity and common LLM failure points.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 38 of its 50 references that are in the map. First posted on arXiv 25 Apr 2024.

Builds on

38 of 50 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  8. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  9. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  10. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  11. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  12. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
26 more
  1. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  2. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  3. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  4. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  5. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
  6. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 299
  7. 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 295
  8. 2023Proving Test Set Contamination in Black Box Language Modelscited by 227
  9. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 218
  10. 2019Probing Neural Network Comprehension of Natural Language Argumentscited by 212
  11. 2023On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspectivecited by 208
  12. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 200
  13. 2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194
  14. 2020Adversarial Filters of Dataset Biasescited by 159
  15. 2024When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboardscited by 134
  16. 2021Robustness Gym: Unifying the NLP Evaluation Landscapecited by 107
  17. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  18. 2023Anchor Points: Benchmarking Models with Much Fewer Examplescited by 84
  19. 2024Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvementscited by 82
  20. 2022GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspectivecited by 72
  21. 2023Mind the instructions: a holistic evaluation of consistency and interactions in prompt-based learningcited by 31
  22. 2023Accuracy and Political Bias of News Source Credibility Ratings by Large Language Modelscited by 25
  23. 2023A Survey on Out-of-Distribution Evaluation of Neural NLP Modelscited by 20
  24. 2023Leveraging Large Language Models for Topic Classification in the Domain of Public Affairscited by 13
  25. 2024Revisiting Word Embeddings in the LLM Eracited by 13
  26. 2023Evaluating Embedding APIs for Information Retrievalcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 10; 2026: 14.

31 papers.

Sort:

By area:

  1. 2024On-Device Language Models: A Comprehensive Reviewcited by 101
  2. 2024What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineeringcited by 88
  3. 2024100 instances is all you need: predicting the success of a new LLM on unseen data by testing on a few instancescited by 24
  4. 2026AgentNoiseBench: Benchmarking Robustness of Tool-Using LLM Agents Under Noisy Conditioncited by 19
  5. 2024Towards Acyclic Preference Evaluation of Language Models via Multiple Evaluatorscited by 13
  6. 2025Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metricscited by 12
  7. 2024SelfPrompt: Autonomously Evaluating LLM Robustness via Domain-Constrained Knowledge Guidelines and Refined Adversarial Promptscited by 10
  8. 2025ReliableEval: A Recipe for Stochastic LLM Evaluation via Method of Momentscited by 10
  9. 2024Shortcut Learning in In-Context Learning: A Surveycited by 8
  10. 2026ErrorMap and ErrorAtlas: Charting the Failure Landscape of Large Language Modelscited by 6
  11. 2026Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environmentscited by 4
  12. 2024KGPA: Robustness Evaluation for Large Language Models via Cross-Domain Knowledge Graphscited by 3
19 more
  1. 2025Adapting Large Language Models for Multi-Domain Retrieval-Augmented-Generationcited by 3
  2. 2025Automated Capability Evaluation of Foundation Modelscited by 3
  3. 2025A Principle-based Framework for the Development and Evaluation of Large Language Models for Health and Wellnesscited by 3
  4. 2026Interactive Evaluation Requires a Design Sciencecited by 3
  5. 2026Can We Trust Item Response Theory for AI Evaluation?cited by 3
  6. 2025Exploring Explanations Improves the Robustness of In-Context Learningcited by 2
  7. 2026LLM Benchmark Datasets Should Be Contamination-Resistantcited by 2
  8. 2026How Many Tasks Are Enough for Agent Benchmark Decisions? A Replay Analysis of Public LLM Agent Benchmarkscited by 2
  9. 2026Robust AI Evaluation through Maximal Lotteriescited by 1
  10. 2026Towards Effective In-context Cross-domain Knowledge Transfer via Domain-invariant-neurons-based Retrievalcited by 1
  11. 2026Reason Analogically via Cross-domain Prior Knowledge: An Empirical Study of Cross-domain Knowledge Transfer for In-Context Learningcited by 1
  12. 2025Surfacing Semantic Orthogonality Across Model Safety Benchmarks: A Multi-Dimensional Analysiscited by 0
  13. 2025Evaluating Prompt Engineering Techniques for Accuracy and Confidence Elicitation in Medical LLMscited by 0
  14. 2025Performance Assessment Strategies for Language Model Applications in Healthcarecited by 0
  15. 2025Correcting Prompt Dependence in LLM Benchmarks: A Bayesian Hierarchical Model with Embedding-Space Clusteringcited by 0
  16. 2026CoDA: Towards Effective Cross-domain Knowledge Transfer via CoT-guided Domain Adaptationcited by 0
  17. 2026When Efficiency Backfires: Cascading LLMs Trigger Cascade Failure under Adversarial Attackcited by 0
  18. 2026Are Near-Tied LLM Rankings Robust to Family-DIF-Guided Benchmark Recomposition?cited by 0
  19. 2026Can We Trust the Judges? Validation of Factuality Evaluation Methods via Answer Perturbationcited by 0