mlsanityData to 7 Oct 2026

Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversations

arXiv 2411.10414 ↗First posted 15 Nov 2024cs.CV cs.CLcited by 118

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

We introduce Llama Guard 3 Vision, a multimodal LLM-based safeguard for human-AI conversations that involves image understanding: it can be used to safeguard content for both multimodal LLM inputs (prompt classification) and outputs (response classification). Unlike the previous text-only Llama Guard versions (Inan et al., 2023; Llama Team, 2024b,a), it is specifically designed to support image reasoning use cases and is optimized to detect harmful multimodal (text and image) prompts and text responses to these prompts. Llama Guard 3 Vision is fine-tuned on Llama 3.2-Vision and demonstrates strong performance on the internal benchmarks using the MLCommons taxonomy. We also test its robustness against adversarial attacks. We believe that Llama Guard 3 Vision serves as a good starting point to build more capable and robust content moderation tools for human-AI conversation with multimodal capabilities.

Key facts. Cited by 118 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 33 references that are in the map. First posted on arXiv 15 Nov 2024.

Builds on

29 of 33 references.

References · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  9. 2013Intriguing properties of neural networkscited by 5,114
  10. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  11. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  12. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
17 more
  1. 2015VQA: Visual Question Answeringcited by 2,489
  2. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  3. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  4. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  5. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  6. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  7. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  8. 2023Are aligned neural networks adversarially aligned?cited by 305
  9. 2023Large Language Models as Tool Makerscited by 293
  10. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 224
  11. 2024Foundational Challenges in Assuring Alignment and Safety of Large Language Modelscited by 193
  12. 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Expertscited by 126
  13. 2024RigorLLM: Resilient Guardrails for Large Language Models against Undesired Contentcited by 65
  14. 2024Introducing v0.5 of the AI Safety Benchmark from MLCommonscited by 60
  15. 2021Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Themcited by 46
  16. 2024Backtracking Improves Generation Safetycited by 31
  17. 2024Gradient-based Jailbreak Images for Multimodal Fusion Modelscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 75; 2026: 43.

118 papers.

Sort:

By area:

  1. 2025GuardReasoner: Towards Reasoning-based LLM Safeguardscited by 89
  2. 2025SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learningcited by 69
  3. 2025Safety in Large Reasoning Models: A Surveycited by 58
  4. 2025Agentic Web: Weaving the Next Web with AI Agentscited by 43
  5. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 38
  6. 2025ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrailscited by 29
  7. 2025Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedbackcited by 27
  8. 2025Security Concerns for Large Language Models: A Surveycited by 26
  9. 2025Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMscited by 25
  10. 2025CultureGuard: Towards Culturally-Aware Dataset and Guard Model for Multilingual Safety Applicationscited by 24
  11. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  12. 2025WebGuard: Building a Generalizable Guardrail for Web Agentscited by 20
38 more
  1. 2025ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Servingcited by 18
  2. 2025Think in Safety: Unveiling and Mitigating Safety Alignment Collapse in Multimodal Large Reasoning Modelcited by 18
  3. 2025Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearningcited by 16
  4. 2025DynaGuard: A Dynamic Guardian Model With User-Defined Policiescited by 16
  5. 2025T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Modelcited by 16
  6. 2025Position: Adversarial ML for LLMs Is Not Making Any Progresscited by 14
  7. 2025Learning to Steer: Input-dependent Steering for Multimodal LLMscited by 13
  8. 2025Shape it Up! Restoring LLM Safety during Finetuningcited by 11
  9. 2025USB: A Comprehensive and Unified Safety Evaluation Benchmark for Multimodal Large Language Modelscited by 10
  10. 2025MSTS: A Multimodal Safety Test Suite for Vision-Language Modelscited by 9
  11. 2025BadVideo: Stealthy Backdoor Attack against Text-to-Video Generationcited by 9
  12. 2025LingoLoop Attack: Trapping MLLMs via Linguistic Context and State Entrapment into Endless Loopscited by 9
  13. 2025Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoringcited by 9
  14. 2026HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmarkcited by 8
  15. 2025ElasticMM: Efficient Multimodal LLMs Serving with Elastic Multimodal Parallelismcited by 7
  16. 2025LionGuard 2: Building Lightweight, Data-Efficient & Localised Multilingual Content Moderatorscited by 7
  17. 2025VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safetycited by 7
  18. 2026FlexGuard: Continuous Risk Scoring for Strictness-Adaptive LLM Content Moderationcited by 7
  19. 2025HoliSafe: Holistic Safety Benchmarking and Modeling for Vision-Language Modelcited by 6
  20. 2025NEXUS: Network Exploration for eXploiting Unsafe Sequences in Multi-Turn LLM Jailbreakscited by 6
  21. 2025Kelp: A Streaming Safeguard for Large Models via Latent Dynamics-Guided Risk Detectioncited by 6
  22. 2026Towards Policy-Adaptive Image Guardrail: Benchmark and Methodcited by 6
  23. 2025Universal Adversarial Attack on Aligned Multimodal LLMscited by 5
  24. 2025ShieldVLM: Safeguarding the Multimodal Implicit Toxicity via Deliberative Reasoning with LVLMscited by 5
  25. 2025Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Modelscited by 5
  26. 2025OMNIGUARD: An Efficient Approach for AI Safety Moderation Across Languages and Modalitiescited by 5
  27. 2025SHIELD: Classifier-Guided Prompting for Robust and Safer LVLMscited by 5
  28. 2025A Survey on Collaborating Small and Large Language Models for Performance, Cost-effectiveness, Cloud-edge Privacy, and Trustworthinesscited by 5
  29. 2025EcoAlign: An Economically Rational Framework for Efficient LVLM Alignmentcited by 5
  30. 2025OmniGuard: Unified Omni-Modal Guardrails with Deliberate Reasoningcited by 5
  31. 2025When Memory Becomes a Vulnerability: Towards Multi-turn Jailbreak Attacks against Text-to-Image Generation Systemscited by 4
  32. 2025CAPTURE: Context-Aware Prompt Injection Testing and Robustness Enhancementcited by 4
  33. 2025Hierarchical Safety Realignment: Lightweight Restoration of Safety in Pruned Large Vision-Language Modelscited by 4
  34. 2025Secure Tug-of-War (SecTOW): Iterative Defense-Attack Training with Reinforcement Learning for Multimodal Model Securitycited by 4
  35. 2025CrossGuard: Safeguarding MLLMs against Joint-Modal Implicit Malicious Attackscited by 4
  36. 2025Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMscited by 4
  37. 2025GuardTrace-VL: Detecting Unsafe Multimodel Reasoning via Iterative Safety Supervisioncited by 4
  38. 2026Visual Exclusivity Attacks: Automatic Multimodal Red Teaming via Agentic Planningcited by 4

50 most cited of 118 All 118 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Back to the Barn with LLAMAs: Evolving Pretrained LLM Backbones in Finetuning Vision Language Models
  2. 2026VLMShield: Efficient and Robust Defense of Vision-Language Models against Malicious Prompts
  3. 2026Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection
  4. 2024Multimodal Adversarial Defense for Vision-Language Models by Leveraging One-To-Many Relationships
  5. 2026To See is Not to Learn: Protecting Multimodal Data from Unauthorized Fine-Tuning of Large Vision-Language Model
  6. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model
  7. 2025Building Trustworthy Multimodal AI: A Review of Fairness, Transparency, and Ethics in Vision-Language Tasks
  8. 2023Making LLaMA SEE and Draw with SEED Tokenizer