ScienceArena: Benchmarking LLMs on Latest Scientific Olympiad Competitions
On the map: LLM reasoning → Mathematical reasoning
Abstract (arXiv)
Benchmark saturation and data contamination increasingly obscure genuine scientific reasoning in frontier LLMs. We introduce textscScienceArena, an olympiad-style benchmark from thirteen public science competitions in physics, chemistry, and biology, including IPhO and IChO 2025--2026, IBO 2023, USAPhO 2026, and USNCO 2025. Its open-ended, multi-step problems use process-credit rubrics, making faithful scoring difficult. We build ScienceArena through an expert-audited digitization pipeline that converts official exams, figures, solutions, and rubrics into structured items verified by olympiad medalists. To scale evaluation beyond costly human grading, we calibrate LLM-as-judge against medalist ground truth on archived answers from five models across IPhO and IChO; two strong judges stay within one point of expert total scores. Medalist notes show that failures often stem from visual grounding, structure fidelity, and global problem control rather than missing terminology. Evaluating fourteen recent LLMs with interleaved solving, we find that top models obtain medal-equivalent rubric scores on several public international exams, while chemistry and long-horizon consistency remain key bottlenecks. We provide an interactive hrefhttps://science-arena.onrender.com/demo.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 15 of its 36 references that are in the map. First posted on arXiv 31 Aug 2026.
Builds on
15 of 36 references.
Closest ancestors · 2
Same direction, newest first
- 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitions29 May
- 2024OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI18 Jun
Other references · 13
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
- 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
- 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 500
- 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Researchcited by 156
- 2025SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Taskscited by 20
- 2025HiPhO: How Far Are (M)LLMs from Humans in the Latest High School Physics Olympiad Benchmark?cited by 19
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs
- 2025EEFSUVA: A New Mathematical Olympiad Benchmark
- 2026PhysElite: How Far Are LLMs from Solving Olympiad-Level Physics Problems?
- 2026SciIF: Benchmarking Scientific Instruction Following Towards Rigorous Scientific Intelligence
- 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models
- 2026FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Tasks
- 2025AMO-Bench: Large Language Models Still Struggle in High School Math Competitions
- 2025Brains vs. Bytes: Evaluating LLM Proficiency in Olympiad Mathematics