MMArch: Benchmarking Multimodal Reasoning Grounded in Architectural Evidence
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Multimodal large language models (MLLMs) perform strongly on engineering imagery, yet existing benchmarks mostly test drawing recognition, information extraction, or compliance checking, leaving open whether models can combine distributed visual evidence with engineering principles to reach a conclusion. We introduce MMArch, a benchmark for architecture and civil engineering spanning ten subdomains and built entirely from figures in peer-reviewed papers. Its 1,212 short-answer items are produced by a decoupled planner--writer pipeline and validated through automated screening, a blind adversarial audit, and expert review, so that answering requires perceiving the relevant evidence, identifying the governing principle, and applying it, not exploiting textual or single-figure shortcuts. Evaluating 18 open-weight and proprietary MLLMs against a domain-expert panel, we find a wide gap: the strongest open-source model attains about 30% and the best proprietary system 52%, while human experts reach 95%, more than forty points ahead. Our error analysis shows that failures concentrate in applying principles and combining evidence across figures rather than in locating it, pointing to substantial headroom for future research. Code and data are available at https://dcx-swjtu.github.io/MMArch/.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 4 of its 30 references that are in the map. First posted on arXiv 10 Aug 2026.
Builds on
4 of 30 references.
References · 4
- 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 314
- 2024MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understandingcited by 23
- 2026AECV-Bench: Benchmarking Multimodal Models on Architectural and Engineering Drawings Understandingcited by 8
- 2026AEC-Bench: A Multimodal Benchmark for Agentic Systems in Architecture, Engineering, and Constructioncited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026DrawingVQA: A Real-World Benchmark for Multi-Depth Visual-Textual Reasoning on Construction Drawings
- 2026MechReason: Benchmarking Multi-Image Multi-Hop Reasoning in Mechanical Engineering
- 2026Can Large Multimodal Models Inspect Buildings? A Hierarchical Benchmark for Structural Pathology Reasoning
- 2025MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problems
- 2026PolyBridgeBench: Benchmarking Multimodal LLMs for Physics-Grounded Bridge Design
- 2025SO-Bench: A Structural Output Evaluation of Multimodal LLMs
- 2025MAC: A Live Benchmark for Multimodal Large Language Models in Scientific Understanding
- 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Models