# DeepPHY: Benchmarking Agentic VLMs on Physical Reasoning

arXiv: https://arxiv.org/abs/2508.05405 · HTML: https://mlsanity.com/paper/2508.05405/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 11 of its 44 references that are in the map.
- First posted on arXiv 7 Aug 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)
- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 11 of 11)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://mlsanity.com/paper/2501.12326/) (2025)
- [UGPhysics: A Comprehensive Benchmark for Undergraduate Physics Reasoning with Large Language Models](https://mlsanity.com/paper/2502.00334/) (2025)
- [PhyX: Does Your Model Have the "Wits" for Physical Reasoning?](https://mlsanity.com/paper/2505.15929/) (2025)
- [SeePhys: Does Seeing Help Thinking? -- Benchmarking Vision-Based Physics Reasoning](https://mlsanity.com/paper/2505.19099/) (2025)
- [A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges](https://mlsanity.com/paper/2403.10249/) (2024)
- [Mobile-R1: Towards Interactive Capability for VLM-Based Mobile Agent via Systematic Training](https://mlsanity.com/paper/2506.20332/) (2025)
- [StarDojo: Benchmarking Open-Ended Behaviors of Agentic Multimodal LLMs in Production-Living Simulations with Stardew Valley](https://mlsanity.com/paper/2507.07445/) (2025)

## Built on by (most cited first, 14 of 14)

- [GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents](https://mlsanity.com/paper/2604.07429/) (2026)
- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [QuantiPhy: A Quantitative Benchmark Evaluating Physical Reasoning Abilities of Vision-Language Models](https://mlsanity.com/paper/2512.19526/) (2025)
- [Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning](https://mlsanity.com/paper/2608.27549/) (2026)
- [Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AI](https://mlsanity.com/paper/2510.04978/) (2025)
- [Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse](https://mlsanity.com/paper/2605.09965/) (2026)
- [SWITCH: Benchmarking Modeling and Handling of Tangible Interfaces in Long-horizon Embodied Scenarios](https://mlsanity.com/paper/2511.17649/) (2025)
- [ICPRL: Acquiring Physical Intuition from Interactive Control](https://mlsanity.com/paper/2603.13295/) (2026)
- [BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs](https://mlsanity.com/paper/2605.30900/) (2026)
- [Discovering High Level Patterns from Simulation Traces](https://mlsanity.com/paper/2602.10009/) (2026)
- [Bridging Physical Reasoning and Task Generalization via Visual Action Outcome Reasoning Alignment](https://mlsanity.com/paper/2607.06522/) (2026)
- [New Evidence, Same Choice: Testing Physical Experiment Selection in Vision Language Models](https://mlsanity.com/paper/2609.11022/) (2026)
- [PhysAI-Bench: A Benchmark for LLM-Based Agentic Decision-Making in Autonomous UAV-Centric Physical AI](https://mlsanity.com/paper/2609.23695/) (2026)
- [Learning from fracture: Teaching language models to design through physical experience](https://mlsanity.com/paper/2610.17804/) (2026)

## Abstract (arXiv)

Although Vision Language Models (VLMs) exhibit strong perceptual abilities and impressive visual reasoning, they struggle with attention to detail and precise action planning in complex, dynamic environments, leading to subpar performance. Real-world tasks typically require complex interactions, advanced spatial reasoning, long-term planning, and continuous strategy refinement, usually necessitating understanding the physics rules of the target scenario. However, evaluating these capabilities in real-world scenarios is often prohibitively expensive. To bridge this gap, we introduce DeepPHY, a novel benchmark framework designed to systematically evaluate VLMs' understanding and reasoning about fundamental physical principles through a series of challenging simulated environments. DeepPHY integrates multiple physical reasoning environments of varying difficulty levels and incorporates fine-grained evaluation metrics. Our evaluation finds that even state-of-the-art VLMs struggle to translate descriptive physical knowledge into precise, predictive control.