Training-Free Agentic Computer Vision for Structural Component Detection in 2D Structural Framing Plans
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Converting structural framing plans into editable finite-element model drafts is labor-intensive and susceptible to transcription errors. Existing building-component recognition systems generally depend on task-specific neural detectors, whereas language-model agents in structural engineering typically operate on text or model data rather than on drawings. To the authors' knowledge, this work is the first to apply an agentic vision-language layer to structural-component detection and model drafting from framing-plan PDFs without task-specific detector training or fine-tuning. A deterministic stage extracts geometric primitives, estimates scale by dimension-ratio consensus, recognizes five entity classes using an explicit drafting grammar, and assembles an editable layout. The agentic stage constrains typed corrections through deterministic candidates, operation-specific admission tests, change-level review, and fail-closed transactions. Evaluation used an author-generated benchmark of 100 plans, divided equally between a development half used for all rule revisions and a seed-disjoint held-out half generated after the rules were frozen and evaluated once. All scores are end-to-end results for the complete framework on the held-out half. Scale estimates were within 0.1% of the generator reference for every drawing. Recall and precision were 0.922/0.997 for columns, 0.886/0.990 for beams, 1.000/1.000 for walls, 1.000/1.000 for braces, and 1.000/0.964 for openings. A controlled study repeated two corruptions three times on three development drawings. Calibration passed all nine trials, whereas member repair satisfied every strict end-state criterion in five of nine trials. Because both benchmark halves share a generator, the evaluation does not address independently drafted plans, raster input, analytical connectivity, or solver validation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 19 of its 50 references that are in the map. First posted on arXiv 18 Aug 2026.
Builds on
19 of 50 references.
Closest ancestors · 1
Same direction, newest first
Other references · 18
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023Let's Verify Step by Stepcited by 3,789
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2021OCR-free Document Understanding Transformercited by 407
- 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
- 2020Neurosymbolic AI: The 3rd Wavecited by 255
- 2024Vision language models are blind: Failing to translate detailed visual features into wordscited by 147
- 2019CubiCasa5K: A Dataset and an Improved Multi-Task Model for Floorplan Image Analysiscited by 50
- 2021FloorPlanCAD: A Large-Scale CAD Drawing Dataset for Panoptic Symbol Spottingcited by 28
6 more
- 2019Deep Floor Plan Recognition Using a Multi-Task Network with Room-Boundary-Guided Attentioncited by 27
- 2023GPT Models in Construction Industry: Opportunities, Limitations, and a Use Case Validationcited by 21
- 2022GAT-CADNet: Graph Attention Network for Panoptic Symbol Spotting in CAD Drawingscited by 13
- 2025Integrating Large Language Models for Automated Structural Analysiscited by 12
- 2024Symbol as Points: Panoptic Symbol Spotting via Point-based Representationcited by 11
- 2025Large Language Model-Driven Code Compliance Checking in Building Information Modelingcited by 10
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Agentic Large Language Models for Automated Structural Analysis of 3D Frame Systems
- 2025A Lightweight Large Language Model-Based Multi-Agent System for 2D Frame Structural Analysis
- 2025A Large Language Model-Empowered Agent for Reliable and Robust Structural Analysis
- 2026BlueprintAgent: Constraint-Triggered Targeted Revisits for Simulation-Ready Generation from Scanned Structural Blueprints
- 2025Large Language Model Agent for Structural Drawing Generation Using ReAct Prompt Engineering and Retrieval Augmented Generation
- 2026From Perception to Autonomous Computational Modeling: A Multi-Agent Approach
- 2026A Novel Multi-Agent Architecture to Reduce Hallucinations of Large Language Models in Multi-Step Structural Modeling
- 2025LVLMs as inspectors: an agentic framework for category-level structural defect annotation