mlsanityData to 7 Oct 2026

When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily Life

arXiv 2601.04043 ↗First posted 7 Jan 2026cs.CLcited by 2

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

As Multimodal Large Language Models (MLLMs) become an indispensable assistant in human life, the unsafe content generated by MLLMs poses a danger to human behavior, perpetually overhanging human society like a sword of Damocles. To investigate and evaluate the safety impact of MLLMs responses on human behavior in daily life, we introduce SaLAD, a multimodal safety benchmark which contains 2,013 real-world image-text samples across 10 common categories, with a balanced design covering both unsafe scenarios and cases of oversensitivity. It emphasizes realistic risk exposure, authentic visual inputs, and fine-grained cross-modal reasoning, ensuring that safety risks cannot be inferred from text alone. We further propose a safety-warning-based evaluation framework that encourages models to provide clear and informative safety warnings, rather than generic refusals. Results on 18 MLLMs demonstrate that the top-performing models achieve a safe response rate of only 57.2% on unsafe queries. Moreover, even popular safety alignment methods limit effectiveness of the models in our scenario, revealing the vulnerabilities of current MLLMs in identifying dangerous behaviors in daily life. Our dataset is available at https://github.com/xinyuelou/SaLAD.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 17 of its 37 references that are in the map. First posted on arXiv 7 Jan 2026.

Builds on

17 of 37 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Towards Evaluating Proactive Risk Awareness of Multimodal Language Models23 May
  2. 2025Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs16 Feb
  3. 2024VLSBench: Unveiling Visual Leakage in Multimodal Safety29 Nov
  4. 2024MOSSBench: Is Your Multimodal Language Model Oversensitive to Safe Queries?22 Jun
  5. 2024Learning To See But Forgetting To Follow: Visual Instruction Tuning Makes LLMs More Prone To Jailbreak Attacks7 May
  6. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks3 Apr
  7. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation14 Mar
  8. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov

Other references · 9

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  4. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  5. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 204
  6. 2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 99
  7. 2025Think in Safety: Unveiling and Mitigating Safety Alignment Collapse in Multimodal Large Reasoning Modelcited by 18
  8. 2025OmniGeo: Towards a Multimodal Large Language Models for Geospatial Artificial Intelligencecited by 7
  9. 2025Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failurescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026TSHA: A Benchmark for Visual Language Models in Trustworthy Safety Hazard Assessment Scenarioscited by 0
  2. 2026SAVOR: Self-Aware Visual Grounding via Confidence-Calibrated Reinforcement Learning for Multimodal Hallucination Mitigationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Multimodal Situational Safety
  2. 2026MME-Safety: A Fine-grained Benchmark for Safety Evaluation of MLLMs
  3. 2025MSTS: A Multimodal Safety Test Suite for Vision-Language Models
  4. 2026The Side Effects of Being Smart: Safety Risks in MLLMs' Multi-Image Reasoning
  5. 2026MTMCS-Bench: Evaluating Contextual Safety of Multimodal Large Language Models in Multi-Turn Dialogues
  6. 2024MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models
  7. 2026When Seeing Overrides Knowing: Visual Dominance and Deferral-Based Method for Personalized Safety in VLMs
  8. 2026CSR-Bench: A Benchmark for Evaluating the Cross-modal Safety and Reliability of MLLMs