mlsanityData to 7 Oct 2026

ARCH-B: Architectural Representation, Comprehension and Hierarchy Benchmark

arXiv 2609.34047 ↗First posted 28 Sep 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Multimodal models increasingly interpret visual environments, but their ability to recognize the same building across photographs, floor plans, elevations, sections, and renderings remains poorly characterized. We introduce ARCH-B, a benchmark of 354 four-choice questions across 11 cross-representational archetypes, constructed from a building-linked corpus of 3.9 million architectural images using visually similar distractors, model-guided difficulty screening, and manual validation. We evaluate 25 multimodal models and collect 5,830 responses from non-expert human participants. Model accuracy ranges from 10.45% to 83.90%, compared with a human baseline of 35.35%. Models perform comparatively well on mixed-representation outlier detection and photograph matching, but remain weaker on floorplan-to-photograph correspondence. Human and model difficulty across archetypes is only weakly correlated (Spearman's (ρ=0.33)). Held-out evaluation confirms that the difficulty identified during screening generalizes beyond the curation models. ARCH-B provides a diagnostic evaluation of visual correspondence and representation transfer across architectural media.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 20 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

16 of 20 references.

References · 16

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  3. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  4. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  5. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  6. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  7. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  8. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  9. 2020Adversarial Filters of Dataset Biasescited by 159
  10. 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 133
  11. 2019CubiCasa5K: A Dataset and an Improved Multi-Task Model for Floorplan Image Analysiscited by 49
  12. 2021FloorPlanCAD: A Large-Scale CAD Drawing Dataset for Panoptic Symbol Spottingcited by 28
4 more
  1. 2026AECV-Bench: Benchmarking Multimodal Models on Architectural and Engineering Drawings Understandingcited by 8
  2. 2024WAFFLE: Multimodal Floorplan Understanding in the Wildcited by 5
  3. 2026ArchSIBench: Benchmarking the Architectural Spatial Intelligence of Vision-Language Modelscited by 2
  4. 2026MMArch: Benchmarking Multimodal Reasoning Grounded in Architectural Evidencecited by 1

Built on by

Not cited yet.