mlsanityData to 14 Oct 2026

SafeBuild-Bench: A Temporal-Robust Construction Safety Benchmark with Graph-Enhanced Data Mining

arXiv 2608.00068 ↗First posted 29 Jul 2026cs.CVcited by 0

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Construction-safety models must handle concrete deployment risks, such as a worker standing near a scaffold edge without guardrails, rather than only recognize common objects in curated images. Yet real inspection archives are redundant, long-tailed, and collected across changing sites and months. We introduce SafeBuild-Bench, a metadata-driven benchmark for evaluating multimodal large language models on construction safety under realistic temporal and site variation. It is mined from 100K+ industrial image-text records and contains 3,314 task instances from over 3,000 expert-verified images, covering multiple-choice hazard identification and free-form hazard description. To make expert verification scalable, we develop GEMS, a graph-enhanced multimodal selection pipeline that combines a proxy-model confusion signal with graph-based diversity to identify informative candidates from redundant streams. On public instruction-tuning data, GEMS-selected subsets preserve robustness-oriented performance under small data budgets. On SafeBuild-Bench, current MLLMs remain far from reliable construction-safety understanding, with the best overall score near 60. We release the benchmark, evaluation scripts, and GEMS codebase at https://github.com/safebuild/gems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 22 of its 42 references that are in the map. First posted on arXiv 29 Jul 2026.

Builds on

22 of 42 references.

References · 22

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  4. 2025Qwen3-VL Technical Reportcited by 2,793
  5. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  6. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  7. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  8. 2025Gemma 3 Technical Reportcited by 1,824
  9. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  10. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  11. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  12. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 579
10 more
  1. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
  2. 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 186
  3. 2022Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Timecited by 90
  4. 2022A Survey on Long-Tailed Visual Recognitioncited by 75
  5. 2024SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflectioncited by 34
  6. 2025RSVLM-QA: A Benchmark Dataset for Remote Sensing Vision Language Model-based Question Answeringcited by 17
  7. 2025Using Vision Language Models for Safety Hazard Identification in Constructioncited by 13
  8. 2024Mastering Collaborative Multi-modal Data Selection: A Focus on Informativeness, Uniqueness, and Representativenesscited by 10
  9. 2026SPICE: Submodular Penalized Information-Conflict Selection for Efficient Large Language Model Trainingcited by 7
  10. 2023Safer Together: Machine Learning Models Trained on Shared Accident Datasets Predict Construction Injuries Better than Company-Specific Modelscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Large model retrieval enhancement framework for construction site risk identification
  2. 2025Automating construction safety inspections using a multi-modal vision-language RAG framework
  3. 2025Prompt to Protection: A Comparative Study of Multimodal LLMs in Construction Hazard Recognition
  4. 2025Automated Hazard Detection in Construction Sites Using Large Language and Vision-Language Models
  5. 2026SafeSceneReason: A Multimodal Reasoning Benchmark Connecting Industrial Hazards with Accident Knowledge
  6. 2024Evaluating Cascaded Methods of Vision-Language Models for Zero-Shot Detection and Association of Hardhats for Increased Construction Safety
  7. 2026Integration of Object Detection and Small VLMs for Construction Safety Hazard Identification
  8. 2024Responsible AI in Construction Safety: Systematic Evaluation of Large Language Models and Prompt Engineering