# Garbage In, Reasoning Out? Why Benchmark Scores are Unreliable and What to Do About It

arXiv: https://arxiv.org/abs/2506.23864 · HTML: https://mlsanity.com/paper/2506.23864/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 13 of its 18 references that are in the map.
- First posted on arXiv 30 Jun 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 13 of 13)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks](https://mlsanity.com/paper/2406.18403/) (2024)
- [Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models](https://mlsanity.com/paper/2305.14763/) (2023)
- [Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space](https://mlsanity.com/paper/2505.15778/) (2025)
- [Learn to Reason Efficiently with Adaptive Length-based Reward Shaping](https://mlsanity.com/paper/2505.15612/) (2025)
- [A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners](https://mlsanity.com/paper/2406.11050/) (2024)
- [Decompose-ToM: Enhancing Theory of Mind Reasoning in Large Language Models through Simulation and Task Decomposition](https://mlsanity.com/paper/2501.09056/) (2025)
- [Self-Contradictory Reasoning Evaluation and Detection](https://mlsanity.com/paper/2311.09603/) (2023)
- [Reassessing Extractive QA Datasets at Scale: LLM-as-a-Judge and In-Depth Analyses](https://mlsanity.com/paper/2504.11972/) (2025)
- [Plausibly Problematic Questions in Multiple-Choice Benchmarks for Commonsense Reasoning](https://mlsanity.com/paper/2410.10854/) (2024)
- [LLMs as Repositories of Factual Knowledge: Limitations and Solutions](https://mlsanity.com/paper/2501.12774/) (2025)
- [Towards Quantifying Commonsense Reasoning with Mechanistic Insights](https://mlsanity.com/paper/2504.10077/) (2025)

## Built on by (most cited first, 5 of 5)

- [When Benchmarks Age: Temporal Misalignment through Large Language Model Factuality Evaluation](https://mlsanity.com/paper/2510.07238/) (2025)
- [Benchmarking the Benchmarks: Testing the Predictive Validity of Commonsense Benchmarks](https://mlsanity.com/paper/2608.03340/) (2026)
- [Frontier AI Forecasting Has a Measurement Problem: An Audit of Progress Evidence](https://mlsanity.com/paper/2608.14903/) (2026)
- [WordPolo: Evaluating Language Models Through Iterative Semantic Feedback](https://mlsanity.com/paper/2609.19006/) (2026)
- [Whose Data, Whose Benchmark? A User-Centric Framework for Auditing Evaluation Resources](https://mlsanity.com/paper/2610.19268/) (2026)

## Abstract (arXiv)

We conduct a systematic audit of three widely used reasoning benchmarks, SocialIQa, FauxPas-EAI, and ToMi, and uncover pervasive flaws in both benchmark items and evaluation methodology. Using five LLMs (GPT-{3, 3.5, 4, o1}, and LLaMA 3.1) as diagnostic tools, we identify structural, semantic, and pragmatic issues in benchmark design (e.g., duplicated items, ambiguous wording, and implausible answers), as well as scoring procedures that prioritize output form over reasoning process. Through systematic human annotation and re-evaluation on cleaned benchmark subsets, we find that model scores often improve not due to due to erratic surface wording variations and not to improved reasoning. Infact, further analyses show that model performance is highly sensitive to minor input variations such as context availability and phrasing, revealing that high scores may reflect alignment with format-specific cues rather than consistent inference based on the input. These findings challenge the validity of current benchmark-based claims about reasoning in LLMs, and highlight the need for evaluation protocols that assess reasoning as a process of drawing inference from available information, rather than as static output selection. We release audited data and evaluation tools to support more interpretable and diagnostic assessments of model reasoning.