ProGuard: Towards Proactive Multimodal Safeguard
Not on the map. Why?
Abstract (arXiv)
The rapid evolution of generative models has led to a continuous emergence of multimodal safety risks, exposing the limitations of existing defense methods. To address these challenges, we propose ProGuard, a vision-language proactive guard that identifies and describes out-of-distribution (OOD) safety risks without the need for model adjustments required by traditional reactive approaches. We first construct a modality-balanced dataset of 87K samples, each annotated with both binary safety labels and risk categories under a hierarchical multimodal safety taxonomy, effectively mitigating modality bias and ensuring consistent moderation across text, image, and text-image inputs. Based on this dataset, we train our vision-language base model purely through reinforcement learning (RL) to achieve efficient and concise reasoning. To approximate proactive safety scenarios in a controlled setting, we further introduce an OOD safety category inference task and augment the RL objective with a synonym-bank-based similarity reward that encourages the model to generate concise descriptions for unseen unsafe categories. Experimental results show that ProGuard achieves performance comparable to closed-source large models on binary safety classification, substantially outperforms existing open-source guard models on unsafe content categorization. Most notably, ProGuard delivers a strong proactive moderation ability, improving OOD risk detection by 52.6% and OOD risk description by 64.8%.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 46 of its 48 references that are in the map. First posted on arXiv 29 Dec 2025.
Builds on
46 of 48 references.
References · 46
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Visual Instruction Tuningcited by 7,869
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2021Zero-Shot Text-to-Image Generationcited by 4,288
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2023A Survey of Large Language Modelscited by 2,576
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
34 more
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
- 2023A Survey on Multimodal Large Language Modelscited by 909
- 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
- 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
- 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 496
- 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 356
- 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
- 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 261
- 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 243
- 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 231
- 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 228
- 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 212
- 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 185
- 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Expertscited by 136
- 2025Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrailscited by 132
- 2024Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversationscited by 129
- 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonablecited by 122
- 2025GuardReasoner: Towards Reasoning-based LLM Safeguardscited by 100
- 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Modelcited by 85
- 2023SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Modelscited by 76
- 2024UnsafeBench: Benchmarking Image Safety Classifiers on Real-World and AI-Generated Imagescited by 53
- 2024LlavaGuard: An Open VLM-based Framework for Safeguarding Vision Datasets and Modelscited by 52
- 2025T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generationcited by 48
- 2024MedSafetyBench: Evaluating and Improving the Medical Safety of Large Language Modelscited by 44
- 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 42
- 2024SafeWatch: An Efficient Safety-Policy Following Video Guardrail Model with Transparent Explanationscited by 38
- 2025Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedbackcited by 30
- 2025DynaGuard: A Dynamic Guardian Model With User-Defined Policiescited by 19
- 2024VLMGuard: Bootstrapping Malicious Prompt Detectors from Unlabeled Vision-Language Prompts in the Wildcited by 18
- 2025RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguardscited by 14
- 2025GSPR: Aligning LLM Safeguards as Generalizable Safety Policy Reasonerscited by 13
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancingcited by 6
- 2026Safe responses matter: Output-aware safety guardrail mitigate over-refusal in MLLMscited by 2
- 2026From Sparse Decisions to Dense Reasoning: A Multi-attribute Trajectory Paradigm for Multimodal Moderationcited by 0
- 2026Yuvion VL: A Multimodal Foundation Model for Adversarial Content and AI Safetycited by 0
- 2026StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancing25 Aug 2026
- 2026Yuvion VL: A Multimodal Foundation Model for Adversarial Content and AI Safety23 Jun 2026
- 2026Safe responses matter: Output-aware safety guardrail mitigate over-refusal in MLLMs20 Jun 2026
- 2026From Sparse Decisions to Dense Reasoning: A Multi-attribute Trajectory Paradigm for Multimodal Moderation28 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2025RedDiffuser: Auditing Multimodal Safety Failures in Vision-Language Models via Reinforced Diffusion
- 2026SafeCap: Improving LVLM Safety with Image Captioning Reinforcement Learning
- 2026SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning
- 2025ConceptGuard: Proactive Safety in Text-and-Image-to-Video Generation through Multimodal Risk Detection
- 2026Towards Policy-Adaptive Image Guardrail: Benchmark and Method
- 2024RapGuard: Safeguarding Multimodal Large Language Models via Rationale-aware Defensive Prompting
- 2026GuardAlign: Test-time Safety Alignment in Multimodal Large Language Models
- 2025Towards Evaluating Proactive Risk Awareness of Multimodal Language Models