# Symbolic or Numerical? Understanding Physics Problem Solving in Reasoning LLMs

arXiv: https://arxiv.org/abs/2507.01334 · HTML: https://mlsanity.com/paper/2507.01334/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 11 of its 21 references that are in the map.
- First posted on arXiv 2 Jul 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 11 of 11)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [CDEval: A Benchmark for Measuring the Cultural Dimensions of Large Language Models](https://mlsanity.com/paper/2311.16421/) (2023)
- [Input-gradient space particle inference for neural network ensembles](https://mlsanity.com/paper/2306.02775/) (2023)
- [An Analysis of User Behaviors for Objectively Evaluating Spoken Dialogue Systems](https://mlsanity.com/paper/2401.04867/) (2024)
- [Uncertainty Quantification and Confidence Intervals for Naive Rare-Event Estimators](https://mlsanity.com/paper/2305.02434/) (2023)
- [Controlling Cloze-test Question Item Difficulty with PLM-based Surrogate Models for IRT Assessment](https://mlsanity.com/paper/2403.01456/) (2024)

## Built on by (most cited first, 3 of 3)

- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AI](https://mlsanity.com/paper/2510.04978/) (2025)
- [Decoupled Physical Modeling and Execution for Physics Reasoning](https://mlsanity.com/paper/2608.22126/) (2026)

## Abstract (arXiv)

Navigating the complexities of physics reasoning has long been a difficult task for Large Language Models (LLMs), requiring a synthesis of profound conceptual understanding and adept problem-solving techniques. In this study, we investigate the application of advanced instruction-tuned reasoning models, such as Deepseek-R1, to address a diverse spectrum of physics problems curated from the challenging SciBench benchmark. Our comprehensive experimental evaluation reveals the remarkable capabilities of reasoning models. Not only do they achieve state-of-the-art accuracy in answering intricate physics questions, but they also generate distinctive reasoning patterns that emphasize on symbolic derivation. Furthermore, our findings indicate that even for these highly sophisticated reasoning models, the strategic incorporation of few-shot prompting can still yield measurable improvements in overall accuracy, highlighting the potential for continued performance gains.