PolyComp: A Polycube-based Benchmark for Compositional 3D Spatial Reasoning in Multimodal Models
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
We introduce PolyComp, a procedurally generated and verified benchmark that stresses visual recognition and compositional spatial reasoning. In each problem, a model must identify which of four options shows a pair of polycube components that can be combined to form a target solid. The benchmark contains 120 problems across four geometry families, and each problem has three different presentation formats using either a single image or multiple images. The random guessing baseline is 25%. Across the three presentations (360 presented problems per model), GPT-5.6 Sol with max effort attains 50.0% accuracy (95% problem-cluster CI 43.3-56.7%) at a mean cost of 0.951 per presented problem, Claude Fable 5 with max effort attains 39.4% (33.1-46.1%) at 0.701, and Gemini 3.1 Pro Preview with thinking level high attains 27.5% (22.8-32.5%), near the 25% random guessing baseline, at 0.350. The observed accuracy spread across geometry families is larger than across presentation formats. We present a problem development and evaluation protocol, cost and token accounting, and release the 120 problems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 19 of its 24 references that are in the map. First posted on arXiv 13 Aug 2026.
Builds on
19 of 24 references.
Closest ancestors · 5
Same direction, newest first
- 2025InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language Models23 Jun
- 2025iVISPAR -- An Interactive Visual-Spatial Reasoning Benchmark for VLMs5 Feb
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
- 2022SQA3D: Situated Question Answering in 3D Scenes14 Oct
- 2022Visual Spatial Reasoning30 Apr
Other references · 14
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
- 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
- 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
- 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
- 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 457
- 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmark
- 2025DynaSolidGeo: A Dynamic Benchmark for Genuine Spatial Mathematical Reasoning of VLMs in Solid Geometry
- 2026Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces
- 2025Spatial457: A Diagnostic Benchmark for 6D Spatial Reasoning of Large Multimodal Models
- 2026GraFT: A Training-Free Framework for Spatial Reasoning in Multimodal Large Language Models via 3D Scene Graphs
- 2026Flame3D: Zero-shot Compositional Reasoning of 3D Scenes with Agentic Language Models
- 2026Cog3DMap: Multi-View Vision-Language Reasoning with 3D Cognitive Maps
- 2025Euclid's Gift: Enhancing Spatial Perception and Reasoning in Vision-Language Models via Geometric Surrogate Tasks