# DynaSolidGeo: A Dynamic Benchmark for Genuine Spatial Mathematical Reasoning of VLMs in Solid Geometry

arXiv: https://arxiv.org/abs/2510.22340 · HTML: https://mlsanity.com/paper/2510.22340/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 41 of its 66 references that are in the map.
- First posted on arXiv 25 Oct 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 30 of 41)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems](https://mlsanity.com/paper/2402.14008/) (2024)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning](https://mlsanity.com/paper/2105.04165/) (2021)
- [LiveBench: A Challenging, Contamination-Limited LLM Benchmark](https://mlsanity.com/paper/2406.19314/) (2024)
- [LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training](https://mlsanity.com/paper/2509.23661/) (2025)
- [Data Contamination: From Memorization to Exploitation](https://mlsanity.com/paper/2203.08242/) (2022)
- [GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning](https://mlsanity.com/paper/2105.14517/) (2021)
- [DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Models](https://mlsanity.com/paper/2411.00836/) (2024)
- [Investigating Data Contamination in Modern Benchmarks for Large Language Models](https://mlsanity.com/paper/2311.09783/) (2023)
- [A Multi-Modal Neural Geometric Solver with Textual Clauses Parsed from Diagram](https://mlsanity.com/paper/2302.11097/) (2023)
- [LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?](https://mlsanity.com/paper/2506.11928/) (2025)
- [An Open Source Data Contamination Report for Large Language Models](https://mlsanity.com/paper/2310.17589/) (2023)
- [GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solving](https://mlsanity.com/paper/2402.10104/) (2024)
- [AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledge](https://mlsanity.com/paper/2412.13670/) (2024)
- [MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmark](https://mlsanity.com/paper/2412.15194/) (2024)
- [Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluation](https://mlsanity.com/paper/2502.17521/) (2025)
- [CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning](https://mlsanity.com/paper/2509.22647/) (2025)
- [A Survey on Data Contamination for Large Language Models](https://mlsanity.com/paper/2502.14425/) (2025)
- [Why Do MLLMs Struggle with Spatial Understanding? A Systematic Analysis from Data to Architecture](https://mlsanity.com/paper/2509.02359/) (2025)
- [GeoSense: Evaluating Identification and Application of Geometric Principles in Multimodal Reasoning](https://mlsanity.com/paper/2504.12597/) (2025)
- [GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training](https://mlsanity.com/paper/2412.11863/) (2024)
- [Defining and Evaluating Visual Language Models' Basic Spatial Abilities: A Perspective from Psychometrics](https://mlsanity.com/paper/2502.11859/) (2025)
- [DSI-Bench: A Benchmark for Dynamic Spatial Intelligence](https://mlsanity.com/paper/2510.18873/) (2025)
- [SOLIDGEO: Measuring Multimodal Spatial Math Reasoning in Solid Geometry](https://mlsanity.com/paper/2505.21177/) (2025)

## Built on by (most cited first, 1 of 1)

- [ScalSelect: Scalable Training-Free Multimodal Data Selection for Efficient Visual Instruction Tuning](https://mlsanity.com/paper/2602.11636/) (2026)

## Abstract (arXiv)

Solid geometry problem solving demands spatial mathematical reasoning that integrates spatial intelligence and symbolic reasoning. However, most existing multimodal mathematical reasoning benchmarks focus primarily on 2D plane geometry, rely on static datasets prone to data contamination and memorization, and evaluate models solely by final answers, overlooking the reasoning process. To address these limitations, we introduce DynaSolidGeo, the first dynamic benchmark for evaluating genuine spatial reasoning in Vision-Language Models (VLMs). Constructed through a semi-automatic annotation pipeline, DynaSolidGeo contains 503 expert-curated seed questions that can, in principle, dynamically generate an unbounded number of diverse multimodal text-visual instances. Beyond answer accuracy, we incorporate process evaluation based on expert-annotated reasoning chains to measure logical validity and causal coherence. Experiments across representative open-source and closed-source VLMs reveal large performance gaps, severe degradation in dynamic settings, and poor performance on tasks requiring high-level spatial intelligence, such as mental rotation and visualization. The code and dataset are available at hrefhttps://zgca-ai4edu.github.io/DynaSolidGeo/DynaSolidGeo.