# HiPhO: How Far Are (M)LLMs from Humans in the Latest High School Physics Olympiad Benchmark?

arXiv: https://arxiv.org/abs/2509.07894 · HTML: https://mlsanity.com/paper/2509.07894/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 9 Sep 2025.

## Builds on (most cited first, 20 of 20)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [Kimi K2: Open Agentic Intelligence](https://mlsanity.com/paper/2507.20534/) (2025)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning](https://mlsanity.com/paper/2507.01006/) (2025)
- [PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Models](https://mlsanity.com/paper/2504.16074/) (2025)
- [Intern-S1: A Scientific Multimodal Foundation Model](https://mlsanity.com/paper/2508.15763/) (2025)
- [PhyX: Does Your Model Have the "Wits" for Physical Reasoning?](https://mlsanity.com/paper/2505.15929/) (2025)
- [SeePhys: Does Seeing Help Thinking? -- Benchmarking Vision-Based Physics Reasoning](https://mlsanity.com/paper/2505.19099/) (2025)
- [Physics: Benchmarking Foundation Models on University-Level Physics Problem Solving](https://mlsanity.com/paper/2503.21821/) (2025)
- [PhysUniBench: A Multi-Modal Physics Reasoning Benchmark at Undergraduate Level](https://mlsanity.com/paper/2506.17667/) (2025)
- [Scaling Physical Reasoning with the PHYSICS Dataset](https://mlsanity.com/paper/2506.00022/) (2025)
- [CMPhysBench: A Benchmark for Evaluating Large Language Models in Condensed Matter Physics](https://mlsanity.com/paper/2508.18124/) (2025)

## Built on by (most cited first, 19 of 19)

- [Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent](https://mlsanity.com/paper/2606.30616/) (2026)
- [P1: Mastering Physics Olympiads with Reinforcement Learning](https://mlsanity.com/paper/2511.13612/) (2025)
- [PhysicsMinions: Winning Gold Medals in the Latest Physics Olympiads with a Coevolutionary Multimodal Multi-Agent System](https://mlsanity.com/paper/2509.24855/) (2025)
- [P1-VL: Bridging Visual Perception and Scientific Reasoning in Physics Olympiads](https://mlsanity.com/paper/2602.09443/) (2026)
- [CausalGame: Benchmarking Causal Thinking of LLM Agents in Games](https://mlsanity.com/paper/2607.04293/) (2026)
- [SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning](https://mlsanity.com/paper/2608.14277/) (2026)
- [Lean4Physics: Comprehensive Reasoning Framework for College-level Physics in Lean4](https://mlsanity.com/paper/2510.26094/) (2025)
- [Towards Reasoning-Preserving Unlearning in Multimodal Large Language Models](https://mlsanity.com/paper/2512.17911/) (2025)
- [OMIBench: Benchmarking Olympiad-Level Multi-Image Reasoning in Large Vision-Language Model](https://mlsanity.com/paper/2604.20806/) (2026)
- [Parametric Skills](https://mlsanity.com/paper/2606.30015/) (2026)
- [Classroom Final Exam: An Instructor-Tested Reasoning Benchmark](https://mlsanity.com/paper/2602.19517/) (2026)
- [TopoAgent: A Self-Evolving Topological Agent for Multimodal Scientific Reasoning](https://mlsanity.com/paper/2607.14658/) (2026)
- [Perfect score on IPhO 2025 theory by Gemini agent](https://mlsanity.com/paper/2603.03352/) (2026)
- [PolyReal: A Benchmark for Real-World Polymer Science Workflows](https://mlsanity.com/paper/2604.02934/) (2026)
- [PhysElite: How Far Are LLMs from Solving Olympiad-Level Physics Problems?](https://mlsanity.com/paper/2608.25097/) (2026)
- [Developing an LLM-Based Feedback System Grounded in Evidence-Centered Design to Support Physics Problem Solving](https://mlsanity.com/paper/2512.10785/) (2025)
- [Can Multimodal Large Language Models Truly Understand Small Objects?](https://mlsanity.com/paper/2604.22884/) (2026)
- [Assessing AI in Introductory Physics Problem Solving](https://mlsanity.com/paper/2607.14303/) (2026)
- [ScienceArena: Benchmarking LLMs on Latest Scientific Olympiad Competitions](https://mlsanity.com/paper/2608.30517/) (2026)

## Abstract (arXiv)

Recently, the physical capabilities of (M)LLMs have garnered increasing attention. However, existing benchmarks for physics suffer from two major gaps: they neither provide systematic and up-to-date coverage of real-world physics competitions such as physics Olympiads, nor enable direct performance comparison with humans. To bridge these gaps, we present HiPhO, the first benchmark dedicated to high school physics Olympiads with human-aligned evaluation. Specifically, HiPhO highlights three key innovations. (1) Comprehensive Data: It compiles 13 latest Olympiad exams from 2024-2025, spanning both international and regional competitions, and covering mixed modalities that encompass problems spanning text-only to diagram-based. (2) Professional Evaluation: We adopt official marking schemes to perform fine-grained grading at both the answer and step level, fully aligned with human examiners to ensure high-quality and domain-specific evaluation. (3) Comparison with Human Contestants: We assign gold, silver, and bronze medals to models based on official medal thresholds, thereby enabling direct comparison between (M)LLMs and human contestants. Our large-scale evaluation of 30 state-of-the-art (M)LLMs shows that: across 13 exams, open-source MLLMs mostly remain at or below the bronze level; open-source LLMs show promising progress with multiple golds; closed-source reasoning MLLMs can achieve 6 to 12 gold medals; and most models still have a significant gap from full marks. These results highlight the performance gap between open-source models and top students, the strong reasoning abilities of closed-source models, and the remaining room for improvement. HiPhO, a human-aligned Olympiad benchmark for multimodal physical reasoning, is open-source at https://github.com/SciYu/HiPhO with a public leaderboard at https://phyarena.github.io/.