# On Robustness and Reliability of Benchmark-Based Evaluation of LLMs

arXiv: https://arxiv.org/abs/2509.04013 · HTML: https://mlsanity.com/paper/2509.04013/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 55 references that are in the map.
- First posted on arXiv 4 Sep 2025.

## Builds on (most cited first, 30 of 34)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [Crowdsourcing Multiple Choice Science Questions](https://mlsanity.com/paper/1707.06209/) (2017)
- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions](https://mlsanity.com/paper/2406.15877/) (2024)
- [QuAC : Question Answering in Context](https://mlsanity.com/paper/1808.07036/) (2018)
- [JudgeLM: Fine-tuned Large Language Models are Scalable Judges](https://mlsanity.com/paper/2310.17631/) (2023)
- [QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization](https://mlsanity.com/paper/2104.05938/) (2021)
- [MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues](https://mlsanity.com/paper/2402.14762/) (2024)
- [ACI-BENCH: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generation](https://mlsanity.com/paper/2306.02022/) (2023)
- [BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices](https://mlsanity.com/paper/2411.12990/) (2024)

## Built on by (most cited first, 23 of 23)

- [AgentNoiseBench: Benchmarking Robustness of Tool-Using LLM Agents Under Noisy Condition](https://mlsanity.com/paper/2602.11348/) (2026)
- [Benchmarks Saturate When The Model Gets Smarter Than The Judge](https://mlsanity.com/paper/2601.19532/) (2026)
- [Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environments](https://mlsanity.com/paper/2605.27209/) (2026)
- [ABCD: All Biases Come Disguised](https://mlsanity.com/paper/2602.17445/) (2026)
- [Safe for Whom? Rethinking How We Evaluate the Safety of LLMs for Real Users](https://mlsanity.com/paper/2512.10687/) (2025)
- [Robustness as an Emergent Property of Task Performance](https://mlsanity.com/paper/2602.03344/) (2026)
- [CLEAR: Revealing How Noise and Ambiguity Degrade Reliability in LLMs for Medicine](https://mlsanity.com/paper/2605.01011/) (2026)
- [GSM-SEM: Benchmark and Framework for Generating Semantically Variant Augmentations](https://mlsanity.com/paper/2605.07053/) (2026)
- [DVD: A Robust Method for Detecting Variant Contamination in Large Language Model Evaluation](https://mlsanity.com/paper/2601.04895/) (2026)
- [Lexara: A User-Centered Toolkit for Evaluating Large Language Models for Conversational Visual Analytics](https://mlsanity.com/paper/2603.05832/) (2026)
- [PRISM: A Benchmark for Programmatic Spatial-Temporal Reasoning](https://mlsanity.com/paper/2605.19382/) (2026)
- [MAVEN: Improving Generalization in Agentic Tool Calling](https://mlsanity.com/paper/2605.30738/) (2026)
- [Same Question, Different Answers: Evaluating LLM Reliability Beyond Accuracy](https://mlsanity.com/paper/2607.22554/) (2026)
- [On Generalization in Agentic Tool Calling: CoreThink Agentic Reasoner and MAVEN Dataset](https://mlsanity.com/paper/2510.22898/) (2025)
- [MACA: A Framework for Distilling Trustworthy LLMs into Efficient Retrievers](https://mlsanity.com/paper/2601.00926/) (2026)
- [Benchmark Health Index: A Systematic Framework for Benchmarking the Benchmarks of LLMs](https://mlsanity.com/paper/2602.11674/) (2026)
- [Simulating Meaning, Nevermore! Introducing ICR: A Semiotic-Hermeneutic Metric for Evaluating Meaning in LLM Text Summaries](https://mlsanity.com/paper/2603.04413/) (2026)
- [StarDrinks: An English and Korean Test Set for SLU Evaluation in a Drink Ordering Scenario](https://mlsanity.com/paper/2604.26500/) (2026)
- [Position: State-of-the-Art Claims Require State-of-the-Art Evidence](https://mlsanity.com/paper/2605.17273/) (2026)
- [Is This Your Final Answer? Cross-Contextual Consistency as a Measure of LLM Credibility](https://mlsanity.com/paper/2608.10315/) (2026)
- [Hallucination-R1: Robustness-Oriented Paraphrase Generation for Factual Consistency](https://mlsanity.com/paper/2609.21227/) (2026)
- [The Copy Ceiling: An Input-Exposure Control for Ontology-Grounded Generation over Curated Corpora](https://mlsanity.com/paper/2609.24885/) (2026)
- [Generalization Is Stability, Not Accuracy: Multi-Axis Evaluation of LLMs](https://mlsanity.com/paper/2610.01428/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) effectiveness is usually evaluated by means of benchmarks such as MMLU, ARC-C, or HellaSwag, where questions are presented in their original wording, thus in a fixed, standardized format. However, real-world applications involve linguistic variability, requiring models to maintain their effectiveness across diverse rewordings of the same question or query. In this study, we systematically assess the robustness of LLMs to paraphrased benchmark questions and investigate whether benchmark-based evaluations provide a reliable measure of model capabilities. We systematically generate various paraphrases of all the questions across six different common benchmarks, and measure the resulting variations in effectiveness of 34 state-of-the-art LLMs, of different size and effectiveness. Our findings reveal that while LLM rankings remain relatively stable across paraphrased inputs, absolute effectiveness scores change, and decline significantly. This suggests that LLMs struggle with linguistic variability, raising concerns about their generalization abilities and evaluation methodologies. Furthermore, the observed performance drop challenges the reliability of benchmark-based evaluations, indicating that high benchmark scores may not fully capture a model's robustness to real-world input variations. We discuss the implications of these findings for LLM evaluation methodologies, emphasizing the need for robustness-aware benchmarks that better reflect practical deployment scenarios.