mlsanityData to 7 Oct 2026

When Harmful Content Gets Camouflaged: Unveiling Perception Failure of LVLMs with CamHarmTI

arXiv 2512.03087 ↗First posted 29 Nov 2025cs.MM cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Large vision-language models (LVLMs) are increasingly used for tasks where detecting multimodal harmful content is crucial, such as online content moderation. However, real-world harmful content is often camouflaged, relying on nuanced text-image interplay, such as memes or images with embedded malicious text, to evade detection. This raises a key question: can LVLMs perceive such camouflaged harmful content as sensitively as humans do? In this paper, we introduce CamHarmTI, a benchmark for evaluating LVLM ability to perceive and interpret camouflaged harmful content within text-image compositions. CamHarmTI consists of over 4,500 samples across three types of image-text posts. Experiments on 100 human users and 12 mainstream LVLMs reveal a clear perceptual gap: humans easily recognize such content (e.g., over 95.75% accuracy), whereas current LVLMs often fail (e.g., ChatGPT-4o achieves only 2.10% accuracy). Moreover, fine-tuning experiments demonstrate that bench serves as an effective resource for improving model perception, increasing accuracy by 55.94% for Qwen2.5VL-7B. Attention analysis and layer-wise probing further reveal that fine-tuning enhances sensitivity primarily in the early layers of the vision encoder, promoting a more integrated scene understanding. These findings highlight the inherent perceptual limitations in LVLMs and offer insight into more human-aligned visual reasoning systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 42 of its 66 references that are in the map. First posted on arXiv 29 Nov 2025.

Builds on

42 of 66 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025On the Reliability of Vision-Language Models Under Adversarial Frequency-Domain Perturbations30 Jul
  2. 2024Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversations15 Nov
  3. 2024TrojVLM: Backdoor Attack Against Vision Language Models28 Sep
  4. 2024Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift27 Jun

Other references · 38

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  7. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  8. 2025Gemma 3 Technical Reportcited by 1,686
  9. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  10. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  11. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  12. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
26 more
  1. 2025Kimi-VL Technical Reportcited by 381
  2. 2024SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehensioncited by 144
  3. 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMscited by 72
  4. 2019Recurrent Neural Networks (RNNs): A gentle Introduction and Overviewcited by 68
  5. 2024MM-Vet v2: A Challenging Benchmark to Evaluate Large Multimodal Models for Integrated Capabilitiescited by 57
  6. 2020SOLID: A Large-Scale Semi-Supervised Dataset for Offensive Language Identificationcited by 53
  7. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  8. 2024Content Moderation by LLM: From Accuracy to Legitimacycited by 30
  9. 2023Adversarial Illusions in Multi-Modal Embeddingscited by 28
  10. 2024A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Taskscited by 27
  11. 2025Analysing the Robustness of Vision-Language-Models to Common Corruptionscited by 16
  12. 2025IllusionCAPTCHA: A CAPTCHA based on Visual Illusioncited by 13
  13. 2024LLMGuard: Guarding Against Unsafe LLM Behaviorcited by 10
  14. 2024Comprehensive Evaluation of Multimodal AI Models in Medical Imaging Diagnosis: From Data Augmentation to Preference-Based Comparisoncited by 10
  15. 2024ICM-Assistant: Instruction-tuning Multimodal Large Language Models for Rule-based Explainable Image Content Moderationcited by 10
  16. 2025Early evidence of how LLMs outperform traditional systems on OCR/HTR tasks for historical recordscited by 10
  17. 2023Multilingual Content Moderation: A Case Study on Redditcited by 8
  18. 2025Religious Bias Landscape in Language and Text-to-Image Models: Analysis, Detection, and Debiasing Strategiescited by 8
  19. 2025Position: Reasoning After Perception Means Reasoning Without Visioncited by 8
  20. 2024Spy-Watermark: Robust Invisible Watermarking for Backdoor Attackcited by 7
  21. 2025PTDiffusion: Free Lunch for Generating Optical Illusion Hidden Pictures with Phase-Transferred Diffusion Modelcited by 7
  22. 2025Unveiling the Lack of LVLM Robustness to Fundamental Visual Variations: Why and Path Forwardcited by 7
  23. 2025Towards Safer Social Media Platforms: Scalable and Performant Few-Shot Harmful Content Moderation Using Large Language Modelscited by 5
  24. 2025A Framework for a Capability-driven Evaluation of Scenario Understanding for Multimodal Large Language Models in Autonomous Drivingcited by 5
  25. 2024Sensitive Content Classification in Social Media: A Holistic Resource and Evaluationcited by 4
  26. 2025Cross-Platform Violence Detection on Social Media: A Dataset and Analysiscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ShieldVLM: Safeguarding the Multimodal Implicit Toxicity via Deliberative Reasoning with LVLMs
  2. 2025Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark Study
  3. 2026From Native Memes to Global Moderation: Cross-Cultural Evaluation of Vision-Language Models for Hateful Meme Detection
  4. 2025Zero-Shot Defense Against Toxic Images via Inherent Multimodal Alignment in LVLMs
  5. 2026They Said Memes Were Harmless-We Found the Ones That Hurt: Decoding Jokes, Symbols, and Cultural References
  6. 2025Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalities
  7. 2026Decodable but Misrouted: Sparse Features Uncover a Readout Gap in Vision-Language Models for Harmful Meme Detection
  8. 2026HarmVideoBench: Benchmarking Harmful Video Understanding in Large Multimodal Models