# PhysicsEval: Inference-Time Techniques to Improve the Reasoning Proficiency of Large Language Models on Physics Problems

arXiv: https://arxiv.org/abs/2508.00079 · HTML: https://mlsanity.com/paper/2508.00079/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 21 of its 60 references that are in the map.
- First posted on arXiv 31 Jul 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 21 of 21)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [Phi-4 Technical Report](https://mlsanity.com/paper/2412.08905/) (2024)
- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI](https://mlsanity.com/paper/2411.04872/) (2024)
- [Cumulative Reasoning with Large Language Models](https://mlsanity.com/paper/2308.04371/) (2023)
- [PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Models](https://mlsanity.com/paper/2504.16074/) (2025)
- [PhysReason: A Comprehensive Benchmark towards Physics-Based Reasoning](https://mlsanity.com/paper/2502.12054/) (2025)
- [UGPhysics: A Comprehensive Benchmark for Undergraduate Physics Reasoning with Large Language Models](https://mlsanity.com/paper/2502.00334/) (2025)
- [Theoretical Physics Benchmark (TPBench) -- a Dataset and Study of AI Reasoning Capabilities in Theoretical Physics](https://mlsanity.com/paper/2502.15815/) (2025)
- [CURIE: Evaluating LLMs On Multitask Scientific Long Context Understanding and Reasoning](https://mlsanity.com/paper/2503.13517/) (2025)
- [FEABench: Evaluating Language Models on Multiphysics Reasoning Ability](https://mlsanity.com/paper/2504.06260/) (2025)
- [Using Large Language Model to Solve and Explain Physics Word Problems Approaching Human Level](https://mlsanity.com/paper/2309.08182/) (2023)
- [Enhancing LLMs for Physics Problem-Solving using Reinforcement Learning with Human-AI Feedback](https://mlsanity.com/paper/2412.06827/) (2024)
- [Testing LLM performance on the Physics GRE: some observations](https://mlsanity.com/paper/2312.04613/) (2023)

## Built on by (most cited first, 7 of 7)

- [CircuitLM: A Multi-Agent LLM-Aided Design Framework for Generating Circuit Schematics from Natural Language Prompts](https://mlsanity.com/paper/2601.04505/) (2026)
- [Courtroom-Style Multi-Agent Debate with Progressive RAG and Role-Switching for Controversial Claim Verification](https://mlsanity.com/paper/2603.28488/) (2026)
- [Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AI](https://mlsanity.com/paper/2510.04978/) (2025)
- [Mastering Olympiad-Level Physics with Artificial Intelligence](https://mlsanity.com/paper/2511.10515/) (2025)
- [AERO: Autonomous Evolutionary Reasoning Optimization via Endogenous Dual-Loop Feedback](https://mlsanity.com/paper/2602.03084/) (2026)
- [LOCA: Logical Chain Augmentation for Scientific Corpus Cleaning](https://mlsanity.com/paper/2510.01249/) (2025)
- [Infinite Problem Generator: Verifiably Scaling Physics Reasoning Data with Agentic Workflows](https://mlsanity.com/paper/2603.14486/) (2026)

## Abstract (arXiv)

The discipline of physics stands as a cornerstone of human intellect, driving the evolution of technology and deepening our understanding of the fundamental principles of the cosmos. Contemporary literature includes some works centered on the task of solving physics problems - a crucial domain of natural language reasoning. In this paper, we evaluate the performance of frontier LLMs in solving physics problems, both mathematical and descriptive. We also employ a plethora of inference-time techniques and agentic frameworks to improve the performance of the models. This includes the verification of proposed solutions in a cumulative fashion by other, smaller LLM agents, and we perform a comparative analysis of the performance that the techniques entail. There are significant improvements when the multi-agent framework is applied to problems that the models initially perform poorly on. Furthermore, we introduce a new evaluation benchmark for physics problems, rm Psmall HYSICSEsmall VAL, consisting of 19,609 problems sourced from various physics textbooks and their corresponding correct solutions scraped from physics forums and educational websites. Our code and data are publicly available at https://github.com/areebuzair/PhysicsEval.