# When Harmful Content Gets Camouflaged: Unveiling Perception Failure of LVLMs with CamHarmTI

arXiv: https://arxiv.org/abs/2512.03087 · HTML: https://mlsanity.com/paper/2512.03087/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 42 of its 66 references that are in the map.
- First posted on arXiv 29 Nov 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)
- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 30 of 42)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Emerging Properties in Unified Multimodal Pretraining](https://mlsanity.com/paper/2505.14683/) (2025)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://mlsanity.com/paper/2005.04790/) (2020)
- [Kimi-VL Technical Report](https://mlsanity.com/paper/2504.07491/) (2025)
- [SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehension](https://mlsanity.com/paper/2404.16790/) (2024)
- [Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversations](https://mlsanity.com/paper/2411.10414/) (2024)
- [MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs](https://mlsanity.com/paper/2411.15296/) (2024)
- [Recurrent Neural Networks (RNNs): A gentle Introduction and Overview](https://mlsanity.com/paper/1912.05911/) (2019)
- [MM-Vet v2: A Challenging Benchmark to Evaluate Large Multimodal Models for Integrated Capabilities](https://mlsanity.com/paper/2408.00765/) (2024)
- [SOLID: A Large-Scale Semi-Supervised Dataset for Offensive Language Identification](https://mlsanity.com/paper/2004.14454/) (2020)
- [Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift](https://mlsanity.com/paper/2406.18844/) (2024)
- [TrojVLM: Backdoor Attack Against Vision Language Models](https://mlsanity.com/paper/2409.19232/) (2024)
- [Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey](https://mlsanity.com/paper/2411.09259/) (2024)
- [Content Moderation by LLM: From Accuracy to Legitimacy](https://mlsanity.com/paper/2409.03219/) (2024)
- [Adversarial Illusions in Multi-Modal Embeddings](https://mlsanity.com/paper/2308.11804/) (2023)
- [A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks](https://mlsanity.com/paper/2411.06284/) (2024)
- [Analysing the Robustness of Vision-Language-Models to Common Corruptions](https://mlsanity.com/paper/2504.13690/) (2025)
- [LLMGuard: Guarding Against Unsafe LLM Behavior](https://mlsanity.com/paper/2403.00826/) (2024)
- [IllusionCAPTCHA: A CAPTCHA based on Visual Illusion](https://mlsanity.com/paper/2502.05461/) (2025)
- [ICM-Assistant: Instruction-tuning Multimodal Large Language Models for Rule-based Explainable Image Content Moderation](https://mlsanity.com/paper/2412.18216/) (2024)
- [Comprehensive Evaluation of Multimodal AI Models in Medical Imaging Diagnosis: From Data Augmentation to Preference-Based Comparison](https://mlsanity.com/paper/2412.05536/) (2024)

## Abstract (arXiv)

Large vision-language models (LVLMs) are increasingly used for tasks where detecting multimodal harmful content is crucial, such as online content moderation. However, real-world harmful content is often camouflaged, relying on nuanced text-image interplay, such as memes or images with embedded malicious text, to evade detection. This raises a key question: can LVLMs perceive such camouflaged harmful content as sensitively as humans do? In this paper, we introduce CamHarmTI, a benchmark for evaluating LVLM ability to perceive and interpret camouflaged harmful content within text-image compositions. CamHarmTI consists of over 4,500 samples across three types of image-text posts. Experiments on 100 human users and 12 mainstream LVLMs reveal a clear perceptual gap: humans easily recognize such content (e.g., over 95.75% accuracy), whereas current LVLMs often fail (e.g., ChatGPT-4o achieves only 2.10% accuracy). Moreover, fine-tuning experiments demonstrate that bench serves as an effective resource for improving model perception, increasing accuracy by 55.94% for Qwen2.5VL-7B. Attention analysis and layer-wise probing further reveal that fine-tuning enhances sensitivity primarily in the early layers of the vision encoder, promoting a more integrated scene understanding. These findings highlight the inherent perceptual limitations in LVLMs and offer insight into more human-aligned visual reasoning systems.