DrawingVQA: A Real-World Benchmark for Multi-Depth Visual-Textual Reasoning on Construction Drawings
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
We introduce DrawingVQA, the first benchmark designed to evaluate multimodal large language models (MLLMs) on real-world construction drawings -- a core media in architecture, civil, and many other engineering practices. Unlike natural images or schematic floor plans, construction drawings fuse abstract geometry, symbolic notation, tabular data, annotations, and domain-specific text, forming a uniquely complex visual-textual domain core to engineering workflows. DrawingVQA bridges this gap with 33 "Issued for Construction" drawings and 92 expertly curated question-answer pairs, spanning three reasoning depths: perceptual understanding, contextual interpretation, and domain-expert reasoning. To evaluate model capabilities, we present a dual categorization framework to jointly analyze performance across seven construction-engineering and four MLLM capability dimensions -- the first to explicitly map engineering workflows to AI reasoning competencies. Evaluations of state-of-the-art MLLMs reveal a substantial gap between model and expert performance, particularly at higher reasoning depths. This benchmark lays a foundation for domain-specialized multimodal reasoning to allow for advancement on integration of AI-driven understanding and real-world engineering workflows.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 14 of its 54 references that are in the map. First posted on arXiv 16 Jul 2026.
Builds on
14 of 54 references.
References · 14
- 2023GPT-4 Technical Reportcited by 18,132
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
- 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
- 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
- 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 233
- 2025See What You Are Told: Visual Attention Sink in Large Multimodal Modelscited by 127
- 2024DesignQA: A Multimodal Benchmark for Evaluating Large Language Models' Understanding of Engineering Documentationcited by 30
- 2025Are Large Pre-trained Vision Language Models Effective Construction Safety Inspectors?cited by 6
- 2025CEQuest: Benchmarking Large Language Models for Construction Estimationcited by 6
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026AECV-Bench: Benchmarking Multimodal Models on Architectural and Engineering Drawings Understanding
- 2026MMArch: Benchmarking Multimodal Reasoning Grounded in Architectural Evidence
- 2026MechVQA: Benchmarking and Enhancing Multimodal LLMs on Comprehensive Mechanical Drawing Understanding
- 2026MechReason: Benchmarking Multi-Image Multi-Hop Reasoning in Mechanical Engineering
- 2026AI vs Human Expert Reasoning: Assessing Agreements in Building Typology Predictions based on Street View Imagery
- 2025A Multi-Stage Hybrid Framework for Automated Interpretation of Multi-View Engineering Drawings Using Vision Language Model
- 2026Do VLMs Reason Like Engineers? A Benchmark and a Stage-wise Evaluation
- 2023From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design