# PhyX: Does Your Model Have the "Wits" for Physical Reasoning?

arXiv: https://arxiv.org/abs/2505.15929 · HTML: https://mlsanity.com/paper/2505.15929/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- First posted on arXiv 21 May 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Built on by (most cited first, 30 of 38)

- [SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning](https://mlsanity.com/paper/2506.01713/) (2025)
- [Look-Back: Implicit Visual Re-focusing in MLLM Reasoning](https://mlsanity.com/paper/2507.03019/) (2025)
- [Step-GUI Technical Report](https://mlsanity.com/paper/2512.15431/) (2025)
- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning](https://mlsanity.com/paper/2507.05255/) (2025)
- [STEP3-VL-10B Technical Report](https://mlsanity.com/paper/2601.09668/) (2026)
- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [PhysUniBench: A Multi-Modal Physics Reasoning Benchmark at Undergraduate Level](https://mlsanity.com/paper/2506.17667/) (2025)
- [DeepPHY: Benchmarking Agentic VLMs on Physical Reasoning](https://mlsanity.com/paper/2508.05405/) (2025)
- [PhysInOne: Visual Physics Learning and Reasoning in One Suite](https://mlsanity.com/paper/2604.09415/) (2026)
- [VisPhyWorld: Probing Physical Reasoning via Code-Driven Video Reconstruction](https://mlsanity.com/paper/2602.13294/) (2026)
- [Cognitive Mismatch in Multimodal Large Language Models for Discrete Symbol Understanding](https://mlsanity.com/paper/2603.18472/) (2026)
- [MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2508.13938/) (2025)
- [Inferring Dynamic Physical Properties from Video Foundation Models](https://mlsanity.com/paper/2510.02311/) (2025)
- [Enhancing Test-Time Scaling of Large Language Models with Hierarchical Retrieval-Augmented MCTS](https://mlsanity.com/paper/2507.05557/) (2025)
- [PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models](https://mlsanity.com/paper/2607.24957/) (2026)
- [Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models](https://mlsanity.com/paper/2604.10219/) (2026)
- [Reward Design for Physical Reasoning in Vision-Language Models](https://mlsanity.com/paper/2604.13993/) (2026)
- [TaxAgent: How Large Language Model Designs Fiscal Policy](https://mlsanity.com/paper/2506.02838/) (2025)
- [DOoM: Difficult Olympiads of Math](https://mlsanity.com/paper/2509.23529/) (2025)
- [PolyReal: A Benchmark for Real-World Polymer Science Workflows](https://mlsanity.com/paper/2604.02934/) (2026)
- [Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models](https://mlsanity.com/paper/2604.14888/) (2026)
- [SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoning](https://mlsanity.com/paper/2605.09266/) (2026)
- [BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs](https://mlsanity.com/paper/2605.30900/) (2026)
- [Apple-π: Benchmarking Thinking with Video Towards Law-Grounded Physical Intelligence](https://mlsanity.com/paper/2607.16401/) (2026)
- [Trace: A Taxonomy-Guided Environment for Multidomain Visual Reasoning](https://mlsanity.com/paper/2607.19790/) (2026)
- [Decoupled Physical Modeling and Execution for Physics Reasoning](https://mlsanity.com/paper/2608.22126/) (2026)
- [PhysElite: How Far Are LLMs from Solving Olympiad-Level Physics Problems?](https://mlsanity.com/paper/2608.25097/) (2026)
- [PhysVista: Benchmarking Physical Intelligence in VLMs via a Perception-Reasoning-Assessment Loop](https://mlsanity.com/paper/2610.00559/) (2026)
- [Intelligent Design 4.0: Paradigm Evolution Toward the Agentic AI Era](https://mlsanity.com/paper/2506.09755/) (2025)

## Abstract (arXiv)

Existing benchmarks fail to capture a crucial aspect of intelligence: physical reasoning, the integrated ability to combine domain knowledge, symbolic reasoning, and understanding of real-world constraints. To address this gap, we introduce PhyX: the first large-scale benchmark designed to assess models capacity for physics-grounded reasoning in visual scenarios. PhyX includes 3K meticulously curated multimodal questions spanning 6 reasoning types across 25 sub-domains and 6 core physics domains: thermodynamics, electromagnetism, mechanics, modern physics, optics, and wave&acoustics. In our comprehensive evaluation, even state-of-the-art models struggle significantly with physical reasoning. GPT-4o, Claude3.7-Sonnet, and GPT-o4-mini achieve only 32.5%, 42.2%, and 45.8% accuracy respectively-performance gaps exceeding 29% compared to human experts. Our analysis exposes critical limitations in current models: over-reliance on memorized disciplinary knowledge, excessive dependence on mathematical formulations, and surface-level visual pattern matching rather than genuine physical understanding. We provide in-depth analysis through fine-grained statistics, detailed case studies, and multiple evaluation paradigms to thoroughly examine physical reasoning capabilities. To ensure reproducibility, we implement a compatible evaluation protocol based on widely-used toolkits such as VLMEvalKit, enabling one-click evaluation. More details are available on our project page: https://phyx-bench.github.io/.