mlsanityData to 14 Oct 2026

PhysReason: A Comprehensive Benchmark towards Physics-Based Reasoning

arXiv 2502.12054 ↗First posted 17 Feb 2025cs.AIcited by 59

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Large language models demonstrate remarkable capabilities across various domains, especially mathematics and logic reasoning. However, current evaluations overlook physics-based reasoning - a complex task requiring physics theorems and constraints. We present PhysReason, a 1,200-problem benchmark comprising knowledge-based (25%) and reasoning-based (75%) problems, where the latter are divided into three difficulty levels (easy, medium, hard). Notably, problems require an average of 8.1 solution steps, with hard requiring 15.6, reflecting the complexity of physics-based reasoning. We propose the Physics Solution Auto Scoring Framework, incorporating efficient answer-level and comprehensive step-level evaluations. Top-performing models like Deepseek-R1, Gemini-2.0-Flash-Thinking, and o3-mini-high achieve less than 60% on answer-level evaluation, with performance dropping from knowledge questions (75.11%) to hard problems (31.95%). Through step-level evaluation, we identified four key bottlenecks: Physics Theorem Application, Physics Process Understanding, Calculation, and Physics Condition Analysis. These findings position PhysReason as a novel and comprehensive benchmark for evaluating physics-based reasoning capabilities in large language models. Our code and data will be published at https:/dxzxy12138.github.io/PhysReason.

Key facts. Cited by 59 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 49 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

30 of 49 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Vision-Language Model-based Physical Reasoning for Robot Liquid Perception10 Apr

Other references · 29

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  6. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  7. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  8. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  9. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  10. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  11. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  12. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
17 more
  1. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  2. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 292
  3. 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 267
  4. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 256
  5. 2023TheoremQA: A Theorem-driven Question Answering datasetcited by 192
  6. 2023Physically Grounded Vision-Language Models for Robotic Manipulationcited by 165
  7. 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understandingcited by 160
  8. 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmarkcited by 134
  9. 2023SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Researchcited by 115
  10. 2023Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Modelscited by 77
  11. 2023Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyondcited by 65
  12. 2024Controllable Text Generation for Large Language Models: A Surveycited by 57
  13. 2023DocMath-Eval: Evaluating Math Reasoning Capabilities of LLMs in Understanding Long and Specialized Documentscited by 45
  14. 2023Forward-Backward Reasoning in Large Language Models for Mathematical Verificationcited by 40
  15. 2023Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Modelscited by 39
  16. 2023DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacycited by 34
  17. 2023Applications of Large Scale Foundation Models for Autonomous Drivingcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 32; 2026: 27.

59 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  2. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  3. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
  4. 2025LEXam: Benchmarking Legal Reasoning on 340 Law Examscited by 54
  5. 2025SeePhys: Does Seeing Help Thinking? -- Benchmarking Vision-Based Physics Reasoningcited by 34
  6. 2026Large Language Model Reasoning Failurescited by 21
  7. 2025PhysUniBench: A Multi-Modal Physics Reasoning Benchmark at Undergraduate Levelcited by 16
  8. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challengescited by 15
  9. 2025MatSciBench: Benchmarking the Reasoning Ability of Large Language Models in Materials Sciencecited by 15
  10. 2025AutoGPS: Automated Geometry Problem Solving via Multimodal Formalization and Deductive Reasoningcited by 14
  11. 2025Let's Reason Formally: Natural-Formal Hybrid Reasoning Enhances LLM's Math Capabilitycited by 11
  12. 2025PhysicsArena: The First Multimodal Physics Reasoning Benchmark Exploring Variable, Process, and Solution Dimensionscited by 9
38 more
  1. 2025Can Theoretical Physics Research Benefit from Language Agents?cited by 9
  2. 2025PhysicsEval: Inference-Time Techniques to Improve the Reasoning Proficiency of Large Language Models on Physics Problemscited by 7
  3. 2025Seeing is Not Reasoning: MVPBench for Graph-based Evaluation of Multi-path Visual Physical CoTcited by 6
  4. 2025CoFFT: Chain of Foresight-Focus Thought for Visual Language Modelscited by 6
  5. 2025Agentic Exploration of Physics Modelscited by 6
  6. 2025CFDLLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamicscited by 5
  7. 2025Lean4Physics: Comprehensive Reasoning Framework for College-level Physics in Lean4cited by 5
  8. 2025EngTrace: A Symbolic Benchmark for Verifiable Process Supervision of Engineering Reasoningcited by 5
  9. 2026OmniFysics: Towards Physical Intelligence Evolution via Omni-Modal Signal Processing and Network Optimizationcited by 5
  10. 2025PhySense: Principle-Based Physics Reasoning Benchmarking for Large Language Modelscited by 4
  11. 2025Mitigating Easy Option Bias in Multiple-Choice Question Answeringcited by 4
  12. 2025PRISM-Physics: Causal DAG-Based Process Evaluation for Physics Reasoningcited by 4
  13. 2025LexGenius: An Expert-Level Benchmark for Large Language Models in Legal General Intelligencecited by 4
  14. 2025From Diagnosis to Improvement: Probing Spatio-Physical Reasoning in Vision Language Modelscited by 3
  15. 2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AIcited by 3
  16. 2026Training and Benchmarking Code Generation for Physics-Inspired Animationscited by 3
  17. 2026OptiVerse: A Comprehensive Benchmark towards Optimization Problem Solvingcited by 3
  18. 2026Causal Scaffolding for Physical Reasoning: A Benchmark for Causally-Informed Physical World Understanding in VLMscited by 3
  19. 2025RAISE: Enhancing Scientific Reasoning in LLMs via Step-by-Step Retrievalcited by 1
  20. 2025DOoM: Difficult Olympiads of Mathcited by 1
  21. 2025Mastering Olympiad-Level Physics with Artificial Intelligencecited by 1
  22. 2026A3-Bench: Benchmarking Memory-Driven Scientific Reasoning via Anchor and Attractor Activationcited by 1
  23. 2026Toward Evaluation Frameworks for Multi-Agent Scientific AI Systemscited by 1
  24. 2026Grading the Unspoken: Evaluating Tacit Reasoning in Quantum Field Theory and String Theory with LLMscited by 1
  25. 2026Fine-Tuning Small Reasoning Models for Quantum Field Theorycited by 1
  26. 2026SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoningcited by 1
  27. 2026Scientific Logicality Enriched Methodology for LLM Reasoning: A Practice in Physicscited by 1
  28. 2026H-GRPO: Permutation-Invariant Reinforcement Learning for Grounded Visual Reasoningcited by 1
  29. 2025SPhyR: Spatial-Physical Reasoning Benchmark on Material Distributioncited by 0
  30. 2025Dissecting Physics Reasoning in Small Language Models: A Multi-Dimensional Analysis from an Educational Perspectivecited by 0
  31. 2025Causes in neuron diagrams, and testing causal reasoning in Large Language Models. A glimpse of the future of philosophy?cited by 0
  32. 2025LOCA: Logical Chain Augmentation for Scientific Corpus Cleaningcited by 0
  33. 2026Towards Efficient and Robust Linguistic Emotion Diagnosis for Mental Health via Multi-Agent Instruction Refinementcited by 0
  34. 2026Discovering High Level Patterns from Simulation Tracescited by 0
  35. 2026SPM-Bench: Benchmarking Large Language Models for Scanning Probe Microscopycited by 0
  36. 2026ARYA: A Physics-Constrained Composable & Deterministic World Model Architecturecited by 0
  37. 2026Dual-Cluster Memory Agent: Resolving Multi-Paradigm Ambiguity in Optimization Problem Solvingcited by 0
  38. 2026PhysNote: Self-Knowledge Notes for Evolvable Physical Reasoning in Vision-Language Modelcited by 0

50 most cited of 59 All 59 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Symbolic or Numerical? Understanding Physics Problem Solving in Reasoning LLMs
  2. 2026PhysMent: An Interactive Approach For LLM Reasoning In Physics Problems
  3. 2025UGPhysics: A Comprehensive Benchmark for Undergraduate Physics Reasoning with Large Language Models
  4. 2025ABench-Physics: Benchmarking Physical Reasoning in LLMs via High-Difficulty and Dynamic Physics Problems
  5. 2025Scaling Physical Reasoning with the PHYSICS Dataset
  6. 2024Physics Reasoner: Knowledge-Augmented Reasoning for Solving Physics Problems with Large Language Models
  7. 2025SymPyBench: A Dynamic Benchmark for Scientific Reasoning with Executable Python Code
  8. 2025PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Models