mlsanityData to 14 Oct 2026

Large model retrieval enhancement framework for construction site risk identification

arXiv 2508.02073 ↗First posted 4 Aug 2025cs.AIcited by 0

On the map: Retrieval-augmented generation → Agentic RAG, agents and memory

Abstract (arXiv)

This study addresses construction site hazard identification by proposing a retrieval-augmented framework that enhances large language models (LLMs) without requiring fine-tuning. Current LLM-based approaches face limitations: image-text matching struggles with complex hazards, while instruction tuning lacks generalization and is resource-intensive. Our method dynamically integrates external knowledge and retrieved similar cases via prompt tuning, overcoming LLMs' limitations in domain knowledge and feature correlation. The framework comprises a case database, an image retrieval module, and an LLM-based reasoning module. Evaluated on real-site data, our approach boosted GLM-4V's accuracy to 50%, a 35.49% improvement over baselines, with consistent gains across hazard types. Ablation studies validated the effectiveness of our image retrieval strategy, showing the superiority of our LPIPS- and CLIP-based method. The proposed technique significantly improves identification accuracy and contextual understanding, demonstrating strong generalization and offering a practical path for intelligent safety risk detection in construction.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation. Builds on 6 of its 28 references that are in the map. First posted on arXiv 4 Aug 2025.

Builds on

6 of 28 references.

References · 6

  1. 2023CLIPSyntel: CLIP and LLM Synergy for Multimodal Question Summarization in Healthcarecited by 43
  2. 2023A study on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program Repaircited by 26
  3. 2024A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoningcited by 25
  4. 2023From CNN to Transformer: A Review of Medical Image Segmentation Modelscited by 22
  5. 2023Can Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining?cited by 21
  6. 2024Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challengescited by 20

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Prompt to Protection: A Comparative Study of Multimodal LLMs in Construction Hazard Recognition
  2. 2025Using Vision Language Models for Safety Hazard Identification in Construction
  3. 2025Automating construction safety inspections using a multi-modal vision-language RAG framework
  4. 2025Automated Hazard Detection in Construction Sites Using Large Language and Vision-Language Models
  5. 2026SafeBuild-Bench: A Temporal-Robust Construction Safety Benchmark with Graph-Enhanced Data Mining
  6. 2024Responsible AI in Construction Safety: Systematic Evaluation of Large Language Models and Prompt Engineering
  7. 2025Are Large Pre-trained Vision Language Models Effective Construction Safety Inspectors?
  8. 2026Integration of Object Detection and Small VLMs for Construction Safety Hazard Identification