# NEWTON: Are Large Language Models Capable of Physical Reasoning?

arXiv: https://arxiv.org/abs/2310.07018 · HTML: https://mlsanity.com/paper/2310.07018/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 10 Oct 2023.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks](https://mlsanity.com/paper/1912.01734/) (2019)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [Dynabench: Rethinking Benchmarking in NLP](https://mlsanity.com/paper/2104.14337/) (2021)
- [ABO: Dataset and Benchmarks for Real-World 3D Object Understanding](https://mlsanity.com/paper/2110.06199/) (2021)
- [Embodied Question Answering](https://mlsanity.com/paper/1711.11543/) (2017)
- [SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference](https://mlsanity.com/paper/1808.05326/) (2018)
- [TidyBot: Personalized Robot Assistance with Large Language Models](https://mlsanity.com/paper/2305.05658/) (2023)
- [IQA: Visual Question Answering in Interactive Environments](https://mlsanity.com/paper/1712.03316/) (2017)
- [PHYRE: A New Benchmark for Physical Reasoning](https://mlsanity.com/paper/1908.05656/) (2019)
- [QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension](https://mlsanity.com/paper/2107.12708/) (2021)
- [PROST: Physical Reasoning of Objects through Space and Time](https://mlsanity.com/paper/2106.03634/) (2021)
- [PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoning](https://mlsanity.com/paper/2112.05136/) (2021)
- [Actionet: An Interactive End-To-End Platform For Task-Based Data Collection And Augmentation In 3D Environment](https://mlsanity.com/paper/2010.01357/) (2020)

## Built on by (most cited first, 30 of 45)

- [Understanding World or Predicting Future? A Comprehensive Survey of World Models](https://mlsanity.com/paper/2411.14499/) (2024)
- [AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation](https://mlsanity.com/paper/2410.00371/) (2024)
- [DrEureka: Language Model Guided Sim-To-Real Transfer](https://mlsanity.com/paper/2406.01967/) (2024)
- [LLM Agents for Education: Advances and Applications](https://mlsanity.com/paper/2503.11733/) (2025)
- [AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation](https://mlsanity.com/paper/2407.07889/) (2024)
- [Octopi: Object Property Reasoning with Large Tactile-Language Models](https://mlsanity.com/paper/2405.02794/) (2024)
- [UGPhysics: A Comprehensive Benchmark for Undergraduate Physics Reasoning with Large Language Models](https://mlsanity.com/paper/2502.00334/) (2025)
- [MacGyver: Are Large Language Models Creative Problem Solvers?](https://mlsanity.com/paper/2311.09682/) (2023)
- [LLMPhy: Parameter-Identifiable Physical Reasoning Combining Large Language Models and Physics Engines](https://mlsanity.com/paper/2411.08027/) (2024)
- [ManipBench: Benchmarking Vision-Language Models for Low-Level Robot Manipulation](https://mlsanity.com/paper/2505.09698/) (2025)
- [REvolve: Reward Evolution with Large Language Models using Human Feedback](https://mlsanity.com/paper/2406.01309/) (2024)
- [LLM-Powered AI Agent Systems and Their Applications in Industry](https://mlsanity.com/paper/2505.16120/) (2025)
- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [Scaling Physical Reasoning with the PHYSICS Dataset](https://mlsanity.com/paper/2506.00022/) (2025)
- [TextToucher: Fine-Grained Text-to-Touch Generation](https://mlsanity.com/paper/2409.05427/) (2024)
- [Physics Context Builders: A Modular Framework for Physical Reasoning in Vision-Language Models](https://mlsanity.com/paper/2412.08619/) (2024)
- [Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models](https://mlsanity.com/paper/2406.16135/) (2024)
- [BuildArena: A Physics-Aligned Interactive Benchmark of LLMs for Engineering Construction](https://mlsanity.com/paper/2510.16559/) (2025)
- [The Need for Verification in AI-Driven Scientific Discovery](https://mlsanity.com/paper/2509.01398/) (2025)
- [PhysGS: Bayesian-Inferred Gaussian Splatting for Physical Property Estimation](https://mlsanity.com/paper/2511.18570/) (2025)
- [Scene Exploration by Vision-Language Models](https://mlsanity.com/paper/2409.17641/) (2024)
- [Seeing is Not Reasoning: MVPBench for Graph-based Evaluation of Multi-path Visual Physical CoT](https://mlsanity.com/paper/2505.24182/) (2025)
- [A Framework for a Capability-driven Evaluation of Scenario Understanding for Multimodal Large Language Models in Autonomous Driving](https://mlsanity.com/paper/2503.11400/) (2025)
- [Advancing AI-Scientist Understanding: Multi-Agent LLMs with Interpretable Physics Reasoning](https://mlsanity.com/paper/2504.01911/) (2025)
- [From EduVisBench to EduVisAgent: A Benchmark and Multi-Agent Framework for Reasoning-Driven Pedagogical Visualization](https://mlsanity.com/paper/2505.16832/) (2025)
- [Conversational Education at Scale: A Multi-LLM Agent Workflow for Procedural Learning and Pedagogic Quality Assessment](https://mlsanity.com/paper/2507.05528/) (2025)
- [EngTrace: A Symbolic Benchmark for Verifiable Process Supervision of Engineering Reasoning](https://mlsanity.com/paper/2511.01650/) (2025)
- [SimuAgent: An LLM-Based Simulink Modeling Assistant Enhanced with Reinforcement Learning](https://mlsanity.com/paper/2601.05187/) (2026)
- [Teaching Physical Awareness to LLMs through Sounds](https://mlsanity.com/paper/2506.08524/) (2025)
- [RobotSmith: Generative Robotic Tool Design for Acquisition of Complex Manipulation Skills](https://mlsanity.com/paper/2506.14763/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs), through their contextualized representations, have been empirically proven to encapsulate syntactic, semantic, word sense, and common-sense knowledge. However, there has been limited exploration of their physical reasoning abilities, specifically concerning the crucial attributes for comprehending everyday objects. To address this gap, we introduce NEWTON, a repository and benchmark for evaluating the physics reasoning skills of LLMs. Further, to enable domain-specific adaptation of this benchmark, we present a pipeline to enable researchers to generate a variant of this benchmark that has been customized to the objects and attributes relevant for their application. The NEWTON repository comprises a collection of 2800 object-attribute pairs, providing the foundation for generating infinite-scale assessment templates. The NEWTON benchmark consists of 160K QA questions, curated using the NEWTON repository to investigate the physical reasoning capabilities of several mainstream language models across foundational, explicit, and implicit reasoning tasks. Through extensive empirical analysis, our results highlight the capabilities of LLMs for physical reasoning. We find that LLMs like GPT-4 demonstrate strong reasoning capabilities in scenario-based tasks but exhibit less consistency in object-attribute reasoning compared to humans (50% vs. 84%). Furthermore, the NEWTON platform demonstrates its potential for evaluating and enhancing language models, paving the way for their integration into physically grounded settings, such as robotic manipulation. Project site: https://newtonreasoning.github.io