SafeWatch: An Efficient Safety-Policy Following Video Guardrail Model with Transparent Explanations
On the map: LLM safety and security → Safety alignment, refusal and guardrails
Abstract (arXiv)
With the rise of generative AI and rapid growth of high-quality video generation, video guardrails have become more crucial than ever to ensure safety and security across platforms. Current video guardrails, however, are either overly simplistic, relying on pure classification models trained on simple policies with limited unsafe categories, which lack detailed explanations, or prompting multimodal large language models (MLLMs) with long safety guidelines, which are inefficient and impractical for guardrailing real-world content. To bridge this gap, we propose SafeWatch, an efficient MLLM-based video guardrail model designed to follow customized safety policies and provide multi-label video guardrail outputs with content-specific explanations in a zero-shot manner. In particular, unlike traditional MLLM-based guardrails that encode all safety policies autoregressively, causing inefficiency and bias, SafeWatch uniquely encodes each policy chunk in parallel and eliminates their position bias such that all policies are attended simultaneously with equal importance. In addition, to improve efficiency and accuracy, SafeWatch incorporates a policy-aware visual token pruning algorithm that adaptively selects the most relevant video tokens for each policy, discarding noisy or irrelevant information. This allows for more focused, policy-compliant guardrail with significantly reduced computational overhead. Considering the limitations of existing video guardrail benchmarks, we propose SafeWatch-Bench, a large-scale video guardrail benchmark comprising over 2M videos spanning six safety categories which covers over 30 tasks to ensure a comprehensive coverage of all potential safety scenarios. SafeWatch outperforms SOTA by 28.2% on SafeWatch-Bench, 13.6% on benchmarks, cuts costs by 10%, and delivers top-tier explanations validated by LLM and human reviews.
Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 48 of its 60 references that are in the map. First posted on arXiv 9 Dec 2024.
Builds on
48 of 60 references.
Closest ancestors · 1
Same direction, newest first
Other references · 47
- 2023GPT-4 Technical Reportcited by 18,396
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
35 more
- 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
- 2022Red Teaming Language Models with Language Modelscited by 1,026
- 2022Token Merging: Your ViT But Fastercited by 1,021
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
- 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
- 2018Real-world Anomaly Detection in Surveillance Videoscited by 577
- 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 513
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 429
- 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
- 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
- 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
- 2020Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervisioncited by 238
- 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
- 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 181
- 2020TransNet V2: An effective deep network architecture for fast shot transition detectioncited by 171
- 2023Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigationscited by 158
- 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 156
- 2021UBnormal: New Benchmark for Supervised Open-Set Video Anomaly Detectioncited by 93
- 2024Harnessing Large Language Models for Training-free Video Anomaly Detectioncited by 87
- 2023Vista-LLaMA: Reducing Hallucination in Video Language Models via Equal Distance to Visual Tokenscited by 71
- 2024MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?cited by 66
- 2024T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Modelscited by 66
- 2024Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLMcited by 64
- 2023PuMer: Pruning and Merging Tokens for Efficient Vision Language Modelscited by 53
- 2020Localizing Anomalies from Weakly-Labeled Videoscited by 52
- 2023Hierarchical Spatio-temporal Decoupling for Text-to-Video Generationcited by 50
- 2024Hawk: Learning to Understand Open-World Video Anomaliescited by 49
- 2024Video Anomaly Detection and Explanation via Large Language Modelscited by 47
- 2022FakeSV: A Multimodal Benchmark with Rich Social Context for Fake News Detection on Short Video Platformscited by 41
- 2023AutoShot: A Short Video Dataset and State-of-the-Art Shot Boundary Detectioncited by 23
- 2024Can Editing LLMs Inject Harm?cited by 22
- 2023In-Contextual Gender Bias Suppression for Large Language Modelscited by 12
- 2022SoK: Content Moderation in Social Media, from Guidelines to Enforcement, and Research to Practicecited by 6
- 2023Malicious or Benign? Towards Effective Content Moderation for Children's Videoscited by 5
Built on by
Citing papers per year: 2025: 16; 2026: 22.
38 papers.
Sort:
By area:
- 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoningcited by 151
- 2025Beyond Reward Hacking: Causal Rewards for Large Language Model Alignmentcited by 42
- 2025The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMscited by 34
- 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 28
- 2025Efficient Multi-modal Large Language Models via Progressive Consistency Distillationcited by 22
- 2025T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Modelscited by 18
- 2025T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Modelcited by 17
- 2025MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generationcited by 16
- 2026A Framework for Formalizing LLM Agent Securitycited by 16
- 2025VidGuard-R1: AI-Generated Video Detection and Explanation via Reasoning MLLMs and RLcited by 15
- 2025Cer-Eval: Certifiable and Cost-Efficient Evaluation Framework for LLMscited by 11
- 2025Debiasing Reward Models by Representation Learning with Guaranteescited by 6
26 more
- 2026Towards Policy-Adaptive Image Guardrail: Benchmark and Methodcited by 6
- 2025Failures to Surface Harmful Contents in Video Large Language Modelscited by 5
- 2025SafeVision: Efficient Image Guardrail with Robust Policy Adherence and Explainabilitycited by 4
- 2025SPARK: Jailbreaking T2V Models by Synergistically Prompting Auditory and Recontextualized Knowledgecited by 4
- 2025ProGuard: Towards Proactive Multimodal Safeguardcited by 4
- 2026Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderationcited by 4
- 2026BLM-Guard: Explainable Multimodal Ad Moderation with Chain-of-Thought and Policy-Aligned Rewardscited by 3
- 2026Two Frames Matter: A Temporal Attack for Text-to-Video Model Jailbreakingcited by 3
- 2026AudioGuard: Toward Comprehensive Audio Safety Protection Across Diverse Threat Modelscited by 3
- 2026EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agentscited by 3
- 2025SENTINEL: A Multi-Level Formal Framework for Safety Evaluation of Foundation Model-based Embodied Agentscited by 2
- 2026GuardReasoner-Omni: A Reasoning-based Multi-modal Guardrail for Text, Image, Video, and Audiocited by 2
- 2026TrajShield: Trajectory-Level Safety Mediation for Defending Text-to-Video Models Against Jailbreak Attackscited by 1
- 2026Design Methodology and Performance Trade-offs Management for Distributed and Compound AI Systemscited by 1
- 2026Pulling The REINS: Training-Free Safety Alignment of Video Diffusion Models via Representation Steeringcited by 1
- 2026EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guardscited by 1
- 2026Latent Space Probing for Adult Content Detection in Video Generative Modelscited by 0
- 2026SafeLens: Deliberate and Efficient Video Guardrails with Fast-and-Slow Screeningcited by 0
- 2026PolicyShiftGuard: Benchmarking and Improving Policy-Adaptive Image Guardrailscited by 0
- 2026From Sports to Safety: Benchmarking Proactive Risk Inference in MLLMscited by 0
- 2026SafeCA: Safe Cross-Attention Localization and Regulation for Text-to-Video Jailbreak Defensecited by 0
- 2026Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generationcited by 0
- 2026Distributed Implicit Harm: A Compositional Safety Blind Spot in MLLM-Based Video Moderationcited by 0
- 2026A Survey on Adversarial Attacks and Defenses for Diffusion Models Across Multiple Modalitiescited by 0
- 2026Trajectory Unlearning on LLM-based Agentscited by 0
- 2026Controllable Multi-label Video Safety Detection via Adaptive Tversky Policy Optimizationcited by 0
- 2026Controllable Multi-label Video Safety Detection via Adaptive Tversky Policy Optimization1 Oct 2026
- 2026Trajectory Unlearning on LLM-based Agents27 Sep 2026
- 2026Distributed Implicit Harm: A Compositional Safety Blind Spot in MLLM-Based Video Moderation31 Aug 2026
- 2026A Survey on Adversarial Attacks and Defenses for Diffusion Models Across Multiple Modalities28 Aug 2026
- 2026Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation27 Aug 2026
- 2026SafeCA: Safe Cross-Attention Localization and Regulation for Text-to-Video Jailbreak Defense11 Aug 2026
- 2026From Sports to Safety: Benchmarking Proactive Risk Inference in MLLMs6 Aug 2026
- 2026PolicyShiftGuard: Benchmarking and Improving Policy-Adaptive Image Guardrails7 Jul 2026
- 2026EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guards30 Jun 2026
- 2026Pulling The REINS: Training-Free Safety Alignment of Video Diffusion Models via Representation Steering15 Jun 2026
- 2026Design Methodology and Performance Trade-offs Management for Distributed and Compound AI Systems12 Jun 2026
- 2026EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agents29 May 2026
26 more
- 2026SafeLens: Deliberate and Efficient Video Guardrails with Fast-and-Slow Screening17 May 2026
- 2026TrajShield: Trajectory-Level Safety Mediation for Defending Text-to-Video Models Against Jailbreak Attacks3 May 2026
- 2026Latent Space Probing for Adult Content Detection in Video Generative Models25 Apr 2026
- 2026AudioGuard: Toward Comprehensive Audio Safety Protection Across Diverse Threat Models10 Apr 2026
- 2026Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderation8 Apr 2026
- 2026A Framework for Formalizing LLM Agent Security19 Mar 2026
- 2026Two Frames Matter: A Temporal Attack for Text-to-Video Model Jailbreaking7 Mar 2026
- 2026Towards Policy-Adaptive Image Guardrail: Benchmark and Method1 Mar 2026
- 2026BLM-Guard: Explainable Multimodal Ad Moderation with Chain-of-Thought and Policy-Aligned Rewards20 Feb 2026
- 2026GuardReasoner-Omni: A Reasoning-based Multi-modal Guardrail for Text, Image, Video, and Audio3 Feb 2026
- 2025ProGuard: Towards Proactive Multimodal Safeguard29 Dec 2025
- 2025SPARK: Jailbreaking T2V Models by Synergistically Prompting Auditory and Recontextualized Knowledge17 Nov 2025
- 2025SafeVision: Efficient Image Guardrail with Robust Policy Adherence and Explainability28 Oct 2025
- 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges27 Oct 2025
- 2025Debiasing Reward Models by Representation Learning with Guarantees27 Oct 2025
- 2025T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Model25 Oct 2025
- 2025SENTINEL: A Multi-Level Formal Framework for Safety Evaluation of Foundation Model-based Embodied Agents14 Oct 2025
- 2025VidGuard-R1: AI-Generated Video Detection and Explanation via Reasoning MLLMs and RL2 Oct 2025
- 2025Efficient Multi-modal Large Language Models via Progressive Consistency Distillation1 Oct 2025
- 2025Failures to Surface Harmful Contents in Video Large Language Models14 Aug 2025
- 2025The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMs15 Jul 2025
- 2025Cer-Eval: Certifiable and Cost-Efficient Evaluation Framework for LLMs2 May 2025
- 2025T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Models22 Apr 2025
- 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar 2025
- 2025MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generation3 Feb 2025
- 2025Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment16 Jan 2025
- 2025T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Modelscited by 18
- 2026Towards Policy-Adaptive Image Guardrail: Benchmark and Methodcited by 6
- 2025Failures to Surface Harmful Contents in Video Large Language Modelscited by 5
- 2025SafeVision: Efficient Image Guardrail with Robust Policy Adherence and Explainabilitycited by 4
- 2025SPARK: Jailbreaking T2V Models by Synergistically Prompting Auditory and Recontextualized Knowledgecited by 4
- 2026AudioGuard: Toward Comprehensive Audio Safety Protection Across Diverse Threat Modelscited by 3
- 2026GuardReasoner-Omni: A Reasoning-based Multi-modal Guardrail for Text, Image, Video, and Audiocited by 2
- 2026TrajShield: Trajectory-Level Safety Mediation for Defending Text-to-Video Models Against Jailbreak Attackscited by 1
- 2026SafeLens: Deliberate and Efficient Video Guardrails with Fast-and-Slow Screeningcited by 0
- 2026PolicyShiftGuard: Benchmarking and Improving Policy-Adaptive Image Guardrailscited by 0
- 2026From Sports to Safety: Benchmarking Proactive Risk Inference in MLLMscited by 0
- 2026SafeCA: Safe Cross-Attention Localization and Regulation for Text-to-Video Jailbreak Defensecited by 0
- 2026SafeCA: Safe Cross-Attention Localization and Regulation for Text-to-Video Jailbreak Defense11 Aug 2026
- 2026From Sports to Safety: Benchmarking Proactive Risk Inference in MLLMs6 Aug 2026
- 2026PolicyShiftGuard: Benchmarking and Improving Policy-Adaptive Image Guardrails7 Jul 2026
- 2026SafeLens: Deliberate and Efficient Video Guardrails with Fast-and-Slow Screening17 May 2026
- 2026TrajShield: Trajectory-Level Safety Mediation for Defending Text-to-Video Models Against Jailbreak Attacks3 May 2026
- 2026AudioGuard: Toward Comprehensive Audio Safety Protection Across Diverse Threat Models10 Apr 2026
- 2026Towards Policy-Adaptive Image Guardrail: Benchmark and Method1 Mar 2026
- 2026GuardReasoner-Omni: A Reasoning-based Multi-modal Guardrail for Text, Image, Video, and Audio3 Feb 2026
- 2025SPARK: Jailbreaking T2V Models by Synergistically Prompting Auditory and Recontextualized Knowledge17 Nov 2025
- 2025SafeVision: Efficient Image Guardrail with Robust Policy Adherence and Explainability28 Oct 2025
- 2025Failures to Surface Harmful Contents in Video Large Language Models14 Aug 2025
- 2025T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Models22 Apr 2025
- 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 28
- 2026A Framework for Formalizing LLM Agent Securitycited by 16
- 2026EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agentscited by 3
- 2025SENTINEL: A Multi-Level Formal Framework for Safety Evaluation of Foundation Model-based Embodied Agentscited by 2
- 2026Trajectory Unlearning on LLM-based Agentscited by 0
- 2026Trajectory Unlearning on LLM-based Agents27 Sep 2026
- 2026EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agents29 May 2026
- 2026A Framework for Formalizing LLM Agent Security19 Mar 2026
- 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges27 Oct 2025
- 2025SENTINEL: A Multi-Level Formal Framework for Safety Evaluation of Foundation Model-based Embodied Agents14 Oct 2025
- 2025Efficient Multi-modal Large Language Models via Progressive Consistency Distillationcited by 22
- 2026Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderationcited by 4
- 2026EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guardscited by 1
- 2026Distributed Implicit Harm: A Compositional Safety Blind Spot in MLLM-Based Video Moderationcited by 0
- 2026Distributed Implicit Harm: A Compositional Safety Blind Spot in MLLM-Based Video Moderation31 Aug 2026
- 2026EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guards30 Jun 2026
- 2026Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderation8 Apr 2026
- 2025Efficient Multi-modal Large Language Models via Progressive Consistency Distillation1 Oct 2025
- 2025T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Modelcited by 17
- 2025Cer-Eval: Certifiable and Cost-Efficient Evaluation Framework for LLMscited by 11
- 2025ProGuard: Towards Proactive Multimodal Safeguardcited by 4
- 2026Design Methodology and Performance Trade-offs Management for Distributed and Compound AI Systemscited by 1
- 2026Latent Space Probing for Adult Content Detection in Video Generative Modelscited by 0
- 2026Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generationcited by 0
- 2026Controllable Multi-label Video Safety Detection via Adaptive Tversky Policy Optimizationcited by 0
- 2026Controllable Multi-label Video Safety Detection via Adaptive Tversky Policy Optimization1 Oct 2026
- 2026Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation27 Aug 2026
- 2026Design Methodology and Performance Trade-offs Management for Distributed and Compound AI Systems12 Jun 2026
- 2026Latent Space Probing for Adult Content Detection in Video Generative Models25 Apr 2026
- 2025ProGuard: Towards Proactive Multimodal Safeguard29 Dec 2025
- 2025T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Model25 Oct 2025
- 2025Cer-Eval: Certifiable and Cost-Efficient Evaluation Framework for LLMs2 May 2025
Similar, not cited
Close in wording, no citation either way
- 2026SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailing
- 2025Guard Vector: Beyond English LLM Guardrails with Task-Vector Composition and Streaming-Aware Prefix SFT
- 2024LlavaGuard: An Open VLM-based Framework for Safeguarding Vision Datasets and Models
- 2026LongGuard: Mechanistic Analysis and Training-Free Mitigation of Long-Context Failure in Safety Guardrails
- 2025OmniGuard: Unified Omni-Modal Guardrails with Deliberate Reasoning
- 2025Protect: Towards Robust Guardrailing Stack for Trustworthy Enterprise LLM Systems
- 2026Triaging Threats to Specialized Guardrails
- 2026BARRED: Synthetic Training of Custom Policy Guardrails via Asymmetric Debate