# Probing Mechanical Reasoning in Large Vision Language Models

arXiv: https://arxiv.org/abs/2410.00318 · HTML: https://mlsanity.com/paper/2410.00318/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 19 of its 85 references that are in the map.
- First posted on arXiv 1 Oct 2024.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 19 of 19)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [Reasoning with Language Model is Planning with World Model](https://mlsanity.com/paper/2305.14992/) (2023)
- [Instruction Tuning with GPT-4](https://mlsanity.com/paper/2304.03277/) (2023)
- [Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning](https://mlsanity.com/paper/2403.16999/) (2024)
- [CogVLM2: Visual Language Models for Image and Video Understanding](https://mlsanity.com/paper/2408.16500/) (2024)
- [SpatialBot: Precise Spatial Understanding with Vision Language Models](https://mlsanity.com/paper/2406.13642/) (2024)
- [Diverse Cotraining Makes Strong Semi-Supervised Segmentor](https://mlsanity.com/paper/2308.09281/) (2023)
- [Demonstrating and Reducing Shortcuts in Vision-Language Representation Learning](https://mlsanity.com/paper/2402.17510/) (2024)

## Built on by (most cited first, 9 of 9)

- [Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning](https://mlsanity.com/paper/2608.27549/) (2026)
- [Physics Context Builders: A Modular Framework for Physical Reasoning in Vision-Language Models](https://mlsanity.com/paper/2412.08619/) (2024)
- [The Philosophical Foundations of Growing AI Like A Child](https://mlsanity.com/paper/2502.10742/) (2025)
- [Video Models Start to Solve Chess, Maze, Sudoku, Mental Rotation, and Raven' Matrices](https://mlsanity.com/paper/2512.05969/) (2025)
- [Vision Language Models Cannot Reason About Physical Transformation](https://mlsanity.com/paper/2603.07109/) (2026)
- [A Multi-Agent Framework Integrating Large Language Models and Generative AI for Accelerated Metamaterial Design](https://mlsanity.com/paper/2503.19889/) (2025)
- [GRAID: Enhancing Spatial Reasoning of VLMs Through High-Fidelity Data Generation](https://mlsanity.com/paper/2510.22118/) (2025)
- [Egocentric Bias in Vision-Language Models](https://mlsanity.com/paper/2602.15892/) (2026)
- [Spatial Competence Benchmark](https://mlsanity.com/paper/2604.09594/) (2026)

## Abstract (arXiv)

Mechanical reasoning is a hallmark of human intelligence, defined by its ubiquitous yet irreplaceable role in human activities ranging from routine tasks to civil engineering. Embedding machines with mechanical reasoning is therefore an important step towards building human-level artificial intelligence. Here, we leveraged 155 cognitive experiments to test the understanding of system stability, gears and pulley systems, leverage principle, inertia and motion, and fluid mechanics in 26 Vision Language Models (VLMs). Results indicate that VLMs consistently perform worse than humans on all domains, while demonstrate significant difficulty in reasoning about gear systems and fluid mechanics. Notably, their performance on these tasks do not improve as number of parameters increase, suggesting that current attention-based architecture may fail to grasp certain underlying mechanisms required for mechanical reasoning, particularly those pertaining to mental simulations.