mlsanityData to 14 Oct 2026

Can Vision-Language Models Assess Proxemic Risk from Egocentric Robot Images?

arXiv 2608.12515 ↗First posted 12 Aug 2026cs.CV cs.ROcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Assessing proxemic danger from a robot's egocentric perspective is critical for safe embodied navigation in human environments and requires both visual and contextual reasoning. We evaluate three opensource vision-language models (VLMs) (textitInternVL, textitQwen-VL, and textitSmolVLM) on the classification of egocentric robot images into four danger levels, comparing three prompting strategies and two rounds of QLoRA fine-tuning against a stratified random baseline. Without fine-tuning, all models perform near the baseline, while fine-tuning yields only modest overall improvements. However, textitQwen-VL with an advanced prompt achieves substantially higher recall for high-danger cases than the other models. An analysis of person localization further shows that correct danger classification does not correspond to better spatial grounding, indicating that a model may produce a useful safety label without attending to the relevant region of the scene. These results show that current VLMs remain limited in fine-grained proxemic reasoning and spatial grounding, although targeted prompting and fine-tuning can improve high-danger detection in selected models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 21 references that are in the map. First posted on arXiv 12 Aug 2026.

Builds on

14 of 21 references.

References · 14

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  8. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  9. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  10. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  11. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  12. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 43
2 more
  1. 2023Chain of Thought Prompt Tuning in Vision Language Modelscited by 34
  2. 2026The Format Taxcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guards
  2. 2026ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models
  3. 2026Contextual Safety Reasoning and Grounding for Open-World Robots
  4. 2024Simultaneous Localization and Affordance Prediction of Tasks from Egocentric Video
  5. 2026HazardArena: Evaluating Semantic Safety in Vision-Language-Action Models
  6. 2026Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanisms
  7. 2026Toward Low-Latency Vision-Language Models with Doubly-Correct Predictions in Egocentric Visual Understanding
  8. 2026SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents