mlsanityData to 14 Oct 2026

AVA-VLM: Adaptive Visual Attention-Vision Language Model for In-the-Wild Construction Site Monitoring

arXiv 2607.05859 ↗First posted 7 Jul 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

Existing construction-site Vision-Language Model (VLM) studies have primarily adapted pretrained VLMs through direct QA-style fine-tuning from a single global image, but we argue that this paradigm remains limited in operational range, reliability under reduced-resolution inputs, and inference efficiency. To address these limitations, we propose AVA-VLM, an Adaptive Visual Attention-Vision Language Model that follows a human-inspired coarse-to-fine strategy: it first reasons over a low-resolution global image and requests a high-resolution local crop only when detailed inspection is needed. We further introduce a region-aware Chain-of-Thought dataset that teaches when to inspect, where to crop, and how to use local evidence. Experiments show that, for violation identification, AVA-VLM improves overall F1 from 62.0 to 75.1 while using only 30.6% of the baseline visual-token budget; for long-distance PPE-violation cases, F1 improves from 16.0 to 63.6. These results demonstrate AVA-VLM's improved robustness to distant and reduced-resolution visual evidence with substantially lower visual-token usage.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 30 of its 46 references that are in the map. First posted on arXiv 7 Jul 2026.

Builds on

30 of 46 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Are Large Pre-trained Vision Language Models Effective Construction Safety Inspectors?14 Aug

Other references · 29

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  5. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  6. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  7. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  8. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
  9. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
  10. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Modelscited by 642
  11. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  12. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 454
17 more
  1. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 313
  2. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reductioncited by 294
  3. 2023MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learningcited by 172
  4. 2023UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformerscited by 54
  5. 2025VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learningcited by 43
  6. 2023Chain of Thought Prompt Tuning in Vision Language Modelscited by 34
  7. 2025Vision-Language Models for Edge Networks: A Comprehensive Surveycited by 29
  8. 2025AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisitioncited by 21
  9. 2025VRU-Accident: A Vision-Language Benchmark for Video Question Answering and Dense Captioning for Accident Scene Understandingcited by 20
  10. 2024Efficient Vision-Language Models by Summarizing Visual Tokens into Compact Registerscited by 19
  11. 2024Privacy-Aware Visual Language Modelscited by 12
  12. 2025Res-Bench: Benchmarking the Robustness of Multimodal Large Language Models to Dynamic Resolution Inputcited by 9
  13. 2025CropVLM: Learning to Zoom for Fine-Grained Vision-Language Perceptioncited by 8
  14. 2022Edge Video Analytics: A Survey on Applications, Systems and Enabling Techniquescited by 7
  15. 2025SELMA: A Speech-Enabled Language Model for Virtual Assistant Interactionscited by 7
  16. 2025Safe-LLaVA: A Privacy-Preserving Vision-Language Dataset and Benchmark for Biometric Safetycited by 7
  17. 20252COOOL: 2nd Workshop on the Challenge Of Out-Of-Label Hazards in Autonomous Drivingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SABRE: Scalable and Automated Benchmarking of VLMs under Stress
  2. 2026A paired synthetic construction-site image dataset for robust computer vision under adverse conditions
  3. 2024Evaluating Cascaded Methods of Vision-Language Models for Zero-Shot Detection and Association of Hardhats for Increased Construction Safety
  4. 2025Using Vision Language Models for Safety Hazard Identification in Construction
  5. 2024Demystifying the Potential of ChatGPT-4 Vision for Construction Progress Monitoring
  6. 2026Look Where It Matters: High-Resolution Crops Retrieval for Efficient VLMs
  7. 2025InspectVLM: Unified in Theory, Unreliable in Practice
  8. 2025Automating construction safety inspections using a multi-modal vision-language RAG framework