mlsanityData to 14 Oct 2026

BlueprintAgent: Constraint-Triggered Targeted Revisits for Simulation-Ready Generation from Scanned Structural Blueprints

arXiv 2609.07362 ↗First posted 7 Sep 2026cs.CL cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Converting in-service reinforced-concrete (RC) building blueprints into simulation-ready models---structured frame representations that support deterministic FEM export and qualified-engineer review---underpins safety assessment and seismic retrofit, but the process remains manual. Direct prompting of a multimodal large language model (MLLM) over a scanned sheet is unreliable: outputs often violate engineering constraints on beam--column support, span count, or 3D continuity. We present BlueprintAgent (BPA), a constraint-triggered multimodal agent for simulation-ready frame extraction from scanned blueprints. BPA treats the MLLM as the primary reader and decision maker, with OCR and computer vision supplying localized evidence. Its central mechanism realizes engineering constraints as callable validators whose entity-level conflict reports trigger targeted MLLM revisits over the local region---an inference-time control distinct from fixed pipelines and free-form self-reflection. We evaluate BPA on 300 real scanned blueprint sheets from 20 anonymized RC frame projects, against five baselines and six ablations. BPA reaches a macro-averaged Beam F1 of 0.994, against 0.301 for single-MLLM zero-shot and 0.820 for a fixed pipeline; removing MLLM-led axis adjudication collapses Beam and Column F1 on complex multi-sheet projects. For dense technical drawings, engineering constraints are best deployed as triggers for entity-level targeted revisits rather than as post-hoc output filters.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 32 references that are in the map. First posted on arXiv 7 Sep 2026.

Builds on

25 of 32 references.

References · 25

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  3. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  4. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
  5. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  6. 2017Lexically Constrained Decoding for Sequence Generation Using Grid Beam Searchcited by 237
  7. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 197
  8. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 183
  9. 2018Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translationcited by 182
  10. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 154
  11. 2022DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysiscited by 148
  12. 2024SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositoriescited by 52
13 more
  1. 2019CubiCasa5K: A Dataset and an Improved Multi-Task Model for Floorplan Image Analysiscited by 50
  2. 2024RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learningcited by 47
  3. 2021FloorPlanCAD: A Large-Scale CAD Drawing Dataset for Panoptic Symbol Spottingcited by 28
  4. 2024Fine-Tuning Vision-Language Model for Automated Engineering Drawing Information Extractioncited by 18
  5. 2025DocLens : A Tool-Augmented Multi-Agent Framework for Long Visual Document Understandingcited by 15
  6. 2025From Drawings to Decisions: A Hybrid Vision-Language Framework for Parsing 2D Engineering Drawings into Structured Manufacturing Knowledgecited by 8
  7. 2025A Multi-Stage Hybrid Framework for Automated Interpretation of Multi-View Engineering Drawings Using Vision Language Modelcited by 6
  8. 2025SlideAgent: Hierarchical Agentic Framework for Multi-Page Visual Document Understandingcited by 3
  9. 2026EngiAI: Capability-Based Evaluation of Tool-Connected LLM Agents for Engineering Designcited by 3
  10. 2024Seg2Act: Global Context-aware Action Generation for Document Logical Structuringcited by 2
  11. 2026How to Build AI Agents by Augmenting LLMs with Codified Human Expert Domain Knowledge? A Software Engineering Frameworkcited by 2
  12. 2026Tokenization Allows Multimodal Large Language Models to Understand, Generate and Edit Architectural Floor Planscited by 1
  13. 2026Image-Based Structural Analysis Using Computer Vision and LLMs: PhotoBeamSolvercited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Training-Free Agentic Computer Vision for Structural Component Detection in 2D Structural Framing Plans
  2. 2026StructureClaw: Traceable LLM Agents and an Executable Benchmark for Structural Engineering Workflows
  3. 2026Handoff-H1: An Orchestrated Vision-Agent System for Material Quantity Takeoff from Construction Blueprints
  4. 2025A Large Language Model-Empowered Agent for Reliable and Robust Structural Analysis
  5. 2026From Perception to Autonomous Computational Modeling: A Multi-Agent Approach
  6. 2025A Lightweight Large Language Model-Based Multi-Agent System for 2D Frame Structural Analysis
  7. 2026VFEAgent: A Multimodal Agent Framework for End-to-End Automated Finite Element Analysis
  8. 2026Self-Improving CAD Generation Agents with Finite Element Analysis as Feedback