mlsanityData to 7 Oct 2026

GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoning

arXiv 2505.11049 ↗First posted 16 May 2025cs.AI cs.CRcited by 38

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

To enhance the safety of VLMs, this paper introduces a novel reasoning-based VLM guard model dubbed GuardReasoner-VL. The core idea is to incentivize the guard model to deliberatively reason before making moderation decisions via online RL. First, we construct GuardReasoner-VLTrain, a reasoning corpus with 123K samples and 631K reasoning steps, spanning text, image, and text-image inputs. Then, based on it, we cold-start our model's reasoning ability via SFT. In addition, we further enhance reasoning regarding moderation through online RL. Concretely, to enhance diversity and difficulty of samples, we conduct rejection sampling followed by data augmentation via the proposed safety-aware data concatenation. Besides, we use a dynamic clipping parameter to encourage exploration in early stages and exploitation in later stages. To balance performance and token efficiency, we design a length-aware safety reward that integrates accuracy, format, and token cost. Extensive experiments demonstrate the superiority of our model. Remarkably, it surpasses the runner-up by 19.27% F1 score on average. We release data, code, and models (3B/7B) of GuardReasoner-VL at https://github.com/yueliu1999/GuardReasoner-VL/

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 83 of its 101 references that are in the map. First posted on arXiv 16 May 2025.

Builds on

83 of 101 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Safety in Large Reasoning Models: A Survey24 Apr
  2. 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable1 Mar
  3. 2025GuardReasoner: Towards Reasoning-based LLM Safeguards30 Jan
  4. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment27 Nov

Other references · 79

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  9. 2023Visual Instruction Tuningcited by 7,792
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  11. 2024Qwen2.5 Technical Reportcited by 4,713
  12. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
38 more
  1. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2023Mistral 7Bcited by 3,065
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  7. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  8. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  9. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  10. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  11. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  12. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  13. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  14. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  15. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  16. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  17. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  18. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
  19. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 515
  20. 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
  21. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
  22. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  23. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  24. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
  25. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
  26. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 364
  27. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  28. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLcited by 244
  29. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 225
  30. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 224
  31. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 224
  32. 2023Mitigating the Alignment Tax of RLHFcited by 181
  33. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 160
  34. 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concretecited by 134
  35. 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Expertscited by 126
  36. 2023Dolphins: Multimodal Language Model for Drivingcited by 124
  37. 2024Visual In-Context Learning for Large Vision-Language Modelscited by 124
  38. 2025Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrailscited by 122

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 21; 2026: 17.

38 papers.

Sort:

By area:

  1. 2025AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?cited by 105
  2. 2025More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Modelscited by 84
  3. 2025Agentic Web: Weaving the Next Web with AI Agentscited by 43
  4. 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agentscited by 42
  5. 2025Backdoor Cleaning without External Guidance in MLLM Fine-tuningcited by 34
  6. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  7. 2025Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimizationcited by 14
  8. 2025DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Modelscited by 11
  9. 2025SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimizationcited by 11
  10. 2026Thinking in Uncertainty: Mitigating Hallucinations in MLRMs with Latent Entropy-Aware Decodingcited by 11
  11. 2025OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluationcited by 10
  12. 2025LingoLoop Attack: Trapping MLLMs via Linguistic Context and State Entrapment into Endless Loopscited by 9
26 more
  1. 2025VisualTrans: A Benchmark for Real-World Visual Transformation Reasoningcited by 9
  2. 2025SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinationscited by 9
  3. 2026EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Modelscited by 5
  4. 2025GeoCAD: Local Geometry-Controllable CAD Generation with Large Language Modelscited by 4
  5. 2025GuardTrace-VL: Detecting Unsafe Multimodel Reasoning via Iterative Safety Supervisioncited by 4
  6. 2026SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoningcited by 4
  7. 2025Thought Purity: A Defense Framework For Chain-of-Thought Attackcited by 3
  8. 2025SDEval: Safety Dynamic Evaluation for Multimodal Large Language Modelscited by 3
  9. 2026Reasoning over Precedents Alongside Statutes: Case-Augmented Deliberative Alignment for LLM Safetycited by 3
  10. 2026WARD: Adversarially Robust Defense of Web Agents Against Prompt Injectionscited by 3
  11. 2025FOCoOp: Enhancing Out-of-Distribution Robustness in Federated Prompt Learning for Vision-Language Modelscited by 2
  12. 2025ProGuard: Towards Proactive Multimodal Safeguardcited by 2
  13. 2026BLM-Guard: Explainable Multimodal Ad Moderation with Chain-of-Thought and Policy-Aligned Rewardscited by 2
  14. 2025Unveiling the Learning Mind of Language Models: A Cognitive Framework and Empirical Studycited by 1
  15. 2025LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Modelscited by 1
  16. 2026Automated Safety Benchmarking: A Multi-agent Pipeline for LVLMscited by 1
  17. 2026RuleSafe-VL: Evaluating Rule-Conditioned Decision Reasoning in Vision-Language Content Moderationcited by 1
  18. 2026Text is All You Need for Vision-Language Model Jailbreakingcited by 0
  19. 2026SaFeR-ToolKit: Structured Reasoning via Virtual Tool Calling for Multimodal Safetycited by 0
  20. 2026V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failurecited by 0
  21. 2026Evolving Safety Landscape of Multi-modal Large Language Models: A Survey of Emerging Threats and Safeguardscited by 0
  22. 2026SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Modelscited by 0
  23. 2026Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Modelscited by 0
  24. 2026Seeing Through Conflicts: Improving Instruction Hierarchy Alignment in Vision-Language Modelscited by 0
  25. 2026ThinkingGuard: Decoding Implicit Hazards via Step-by-Step Risk Attribution in Multimodal Large Language Modelscited by 0
  26. 2026Controllable Multi-label Video Safety Detection via Adaptive Tversky Policy Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025VLMGuard-R1: Proactive Safety Alignment for VLMs via Reasoning-Driven Prompt Optimization
  2. 2025Safety Through Reasoning: An Empirical Study of Reasoning Guardrail Models
  3. 2026Robust and Efficient Guardrails with Latent Reasoning
  4. 2026DT-Guard: Intent-Driven Reasoning-Active Training for Reasoning-Free LLM Safety Guardrail
  5. 2024R2-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoning
  6. 2026Do Safety Guardrails Need to Reason? LeanGuard: A Fast and Light Approach for Robust Moderation
  7. 2026LatentGuard: Efficient and Inspectable Latent Reasoning for LLM Safeguards
  8. 2025ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Moments