mlsanityData to 7 Oct 2026

Evaluating Cascaded Methods of Vision-Language Models for Zero-Shot Detection and Association of Hardhats for Increased Construction Safety

arXiv 2410.12225 ↗First posted 16 Oct 2024cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

This paper evaluates the use of vision-language models (VLMs) for zero-shot detection and association of hardhats to enhance construction safety. Given the significant risk of head injuries in construction, proper enforcement of hardhat use is critical. We investigate the applicability of foundation models, specifically OWLv2, for detecting hardhats in real-world construction site images. Our contributions include the creation of a new benchmark dataset, Hardhat Safety Detection Dataset, by filtering and combining existing datasets and the development of a cascaded detection approach. Experimental results on 5,210 images demonstrate that the OWLv2 model achieves an average precision of 0.6493 for hardhat detection. We further analyze the limitations and potential improvements for real-world applications, highlighting the strengths and weaknesses of current foundation models in safety perception domains.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 9 of its 20 references that are in the map. First posted on arXiv 16 Oct 2024.

Builds on

9 of 20 references.

References · 9

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2018Object Detection with Deep Learning: A Reviewcited by 524
  3. 2023Scaling Open-Vocabulary Object Detectioncited by 335
  4. 2024Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Drivingcited by 52
  5. 2024Towards Explainable, Safe Autonomous Driving with Language Embeddings for Novelty Identification and Active Learning: Framework and Experimental Analysis with Real-World Data Setscited by 25
  6. 2024ActiveAnno3D -- An Active Learning Framework for Multi-Modal 3D Object Detectioncited by 17
  7. 2024Language-Driven Active Learning for Diverse Open-Set 3D Object Detectioncited by 15
  8. 2024Evaluating Vision-Language Models for Zero-Shot Detection, Classification, and Association of Motorcycles, Passengers, and Helmetscited by 4
  9. 2024Driver Activity Classification Using Generalizable Representations from Vision-Language Modelscited by 3

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Finding the Reflection Point: Unpadding Images to Remove Data Augmentation Artifacts in Large Open Source Image Datasets for Machine Learningcited by 0
  2. 2025Beyond General Prompts: Automated Prompt Refinement using Contrastive Class Alignment Scores for Disambiguating Objects in Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Are Large Pre-trained Vision Language Models Effective Construction Safety Inspectors?
  2. 2026Integration of Object Detection and Small VLMs for Construction Safety Hazard Identification
  3. 2025Using Vision Language Models for Safety Hazard Identification in Construction
  4. 2025Automating construction safety inspections using a multi-modal vision-language RAG framework
  5. 2025Prompt to Protection: A Comparative Study of Multimodal LLMs in Construction Hazard Recognition
  6. 2025Automated Hazard Detection in Construction Sites Using Large Language and Vision-Language Models
  7. 2026AVA-VLM: Adaptive Visual Attention-Vision Language Model for In-the-Wild Construction Site Monitoring
  8. 2026SafeBuild-Bench: A Temporal-Robust Construction Safety Benchmark with Graph-Enhanced Data Mining