# Safe Semantics, Unsafe Interpretations: Tackling Implicit Reasoning Safety in Large Vision-Language Models

arXiv: https://arxiv.org/abs/2508.08926 · HTML: https://mlsanity.com/paper/2508.08926/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 6 of its 14 references that are in the map.
- First posted on arXiv 12 Aug 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)

## Builds on (most cited first, 6 of 6)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [Mimic In-Context Learning for Multimodal Tasks](https://mlsanity.com/paper/2504.08851/) (2025)
- [Egoshots, an ego-vision life-logging dataset and semantic fidelity metric to evaluate diversity in image captioning models](https://mlsanity.com/paper/2003.11743/) (2020)

## Built on by (most cited first, 3 of 3)

- [Aetheria: A multimodal interpretable content safety framework based on multi-agent debate and collaboration](https://mlsanity.com/paper/2512.02530/) (2025)
- [Visual Attention Reasoning via Hierarchical Search and Self-Verification](https://mlsanity.com/paper/2510.18619/) (2025)
- [Distributed Implicit Harm: A Compositional Safety Blind Spot in MLLM-Based Video Moderation](https://mlsanity.com/paper/2609.00206/) (2026)

## Abstract (arXiv)

Large Vision-Language Models face growing safety challenges with multimodal inputs. This paper introduces the concept of Implicit Reasoning Safety, a vulnerability in LVLMs. Benign combined inputs trigger unsafe LVLM outputs due to flawed or hidden reasoning. To showcase this, we developed Safe Semantics, Unsafe Interpretations, the first dataset for this critical issue. Our demonstrations show that even simple In-Context Learning with SSUI significantly mitigates these implicit multimodal threats, underscoring the urgent need to improve cross-modal implicit reasoning.