BrainBench: Exposing the Commonsense Reasoning Gap in Large Language Models
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Large language models (LLMs) achieve impressive scores on standard benchmarks yet routinely fail questions that any human would answer correctly in seconds. We introduce BrainBench, a benchmark of 100 brainteaser questions spanning 20 carefully designed categories, each targeting a specific commonsense reasoning failure mode in LLMs. Categories range from implicit physical constraints ("Should I walk or drive my rental car to the return lot?") to semantic scope tricks and default assumption hijacks. We evaluate eight frontier models -- four from the Claude family and four from the GPT family -- using a zero-shot protocol with 10 independent runs per question. The best model, Claude Opus 4.6 with extended thinking, achieves only 80.3% accuracy; the worst, GPT-4o, scores 39.7%. Even top-performing models exhibit a 6-16 percentage-point gap between accuracy and consistency, revealing stochastic reasoning. Cross-lingual evaluation in Chinese shows most models degrade by 2-8 percentage points, confirming that these failures reflect reasoning deficits rather than language-specific artifacts. BrainBench provides a fine-grained diagnostic tool for identifying where and why LLMs substitute surface heuristics for genuine commonsense reasoning.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 9 of its 18 references that are in the map. First posted on arXiv 16 Mar 2026.
Builds on
9 of 18 references.
References · 9
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
- 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
- 2020The Next Decade in AI: Four Steps Towards Robust Artificial Intelligencecited by 195
- 2021UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmarkcited by 98
- 2021RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledgecited by 61
- 2024SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sensecited by 6
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025HellaSwag-Pro: A Large-Scale Bilingual Benchmark for Evaluating the Robustness of LLMs in Commonsense Reasoning
- 2025Benchmarking Chinese Commonsense Reasoning with a Multi-hop Reasoning Perspective
- 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
- 2023BRAINTEASER: Lateral Thinking Puzzles for Large Language Models
- 2025SCoRE: Benchmarking Long-Chain Reasoning in Commonsense Scenarios
- 2024LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models
- 2024Linguini: A benchmark for language-agnostic linguistic reasoning
- 2024metabench -- A Sparse Benchmark of Reasoning and Knowledge in Large Language Models