mlsanityData to 7 Oct 2026

T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Models

arXiv 2407.05965 ↗First posted 8 Jul 2024cs.CV cs.AI cs.CLcited by 66

Not on the map. Why?

Cited by 29 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The recent development of Sora leads to a new era in text-to-video (T2V) generation. Along with this comes the rising concern about its security risks. The generated videos may contain illegal or unethical content, and there is a lack of comprehensive quantitative understanding of their safety, posing a challenge to their reliability and practical deployment. Previous evaluations primarily focus on the quality of video generation. While some evaluations of text-to-image models have considered safety, they cover fewer aspects and do not address the unique temporal risk inherent in video generation. To bridge this research gap, we introduce T2VSafetyBench, a new benchmark designed for conducting safety-critical assessments of text-to-video models. We define 12 critical aspects of video generation safety and construct a malicious prompt dataset including real-world prompts, LLM-generated prompts and jailbreak attack-based prompts. Based on our evaluation results, we draw several important findings, including: 1) no single model excels in all aspects, with different models showing various strengths; 2) the correlation between GPT-4 assessments and manual reviews is generally high; 3) there is a trade-off between the usability and safety of text-to-video generative models. This indicates that as the field of video generation rapidly advances, safety risks are set to surge, highlighting the urgency of prioritizing video safety. We hope that T2VSafetyBench can provide insights for better understanding the safety of video generation in the era of generative AI.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Builds on 50 of its 68 references that are in the map. First posted on arXiv 8 Jul 2024.

Builds on

50 of 68 references.

References · 50

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  7. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  8. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  9. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  10. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  11. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  12. 2018Datasheets for Datasetscited by 1,371
38 more
  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  3. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  4. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  5. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  6. 2023Erasing Concepts from Diffusion Modelscited by 530
  7. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  8. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  9. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  10. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  11. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  12. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 385
  13. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  14. 2022Red-Teaming the Stable Diffusion Safety Filtercited by 257
  15. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 255
  16. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
  17. 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?cited by 239
  18. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  19. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 208
  20. 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 189
  21. 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 184
  22. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 171
  23. 2023Holistic Evaluation of Text-To-Image Modelscited by 160
  24. 2024Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Modelscited by 159
  25. 2024VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Modelscited by 124
  26. 2023FETV: A Benchmark for Fine-Grained Evaluation of Open-Domain Text-to-Video Generationcited by 119
  27. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  28. 2022Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generationcited by 90
  29. 2023Identifying and Mitigating the Security Risks of Generative AIcited by 90
  30. 2022GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspectivecited by 72
  31. 2024Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Modelscited by 68
  32. 2023Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Modelscited by 56
  33. 2024UnsafeBench: Benchmarking Image Safety Classifiers on Real-World and AI-Generated Imagescited by 52
  34. 2024MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Modelscited by 51
  35. 2023MovieFactory: Automatic Movie Creation from Text using Large Generative Models for Language and Imagescited by 42
  36. 2024Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfelscited by 38
  37. 2024B-AVIBench: Towards Evaluating the Robustness of Large Vision-Language Model on Black-box Adversarial Visual-Instructionscited by 26
  38. 2024BSPA: Exploring Black-box Stealthy Prompt Attacks against Image Generatorscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 30; 2026: 30.

66 papers.

Sort:

By area:

  1. 2024SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generationcited by 125
  2. 2025VideoRAG: Retrieval-Augmented Generation over Video Corpuscited by 55
  3. 2024Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedbackcited by 45
  4. 2024SafeWatch: An Efficient Safety-Policy Following Video Guardrail Model with Transparent Explanationscited by 37
  5. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  6. 2024The Dawn of Video Generation: Preliminary Explorations with SORA-like Modelscited by 30
  7. 2025VPO: Aligning Text-to-Video Generation Models with Prompt Optimizationcited by 27
  8. 2025CineTechBench: A Benchmark for Cinematographic Technique Understanding and Generationcited by 21
  9. 2025T2V-OptJail: Discrete Prompt Optimization for Text-to-Video Jailbreak Attackscited by 18
  10. 2025T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Modelscited by 17
  11. 2025T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Modelcited by 16
  12. 2025Survey of Adversarial Robustness in Multimodal Large Language Modelscited by 14
38 more
  1. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applicationscited by 14
  2. 2025DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarioscited by 14
  3. 2025VideoEraser: Concept Erasure in Text-to-Video Diffusion Modelscited by 12
  4. 2025Video Unlearning via Low-Rank Refusal Vectorcited by 11
  5. 2025Can You Count to Nine? A Human Evaluation Benchmark for Counting Limits in Modern Text-to-Video Modelscited by 10
  6. 2025Jailbreaking on Text-to-Video Models via Scene Splitting Strategycited by 10
  7. 2025BadVideo: Stealthy Backdoor Attack against Text-to-Video Generationcited by 9
  8. 2026When Safety Collides: Resolving Multi-Category Harmful Conflicts in Text-to-Image Diffusion via Adaptive Safety Guidancecited by 9
  9. 2025Red-Teaming Text-to-Image Systems by Rule-based Preference Modelingcited by 8
  10. 2024Autonomous LLM-Enhanced Adversarial Attack for Text-to-Motioncited by 7
  11. 2025GRADEO: Towards Human-Like Evaluation for Text-to-Video Generation via Multi-Step Reasoningcited by 7
  12. 2025RunawayEvil: Jailbreaking the Image-to-Video Generative Modelscited by 7
  13. 2025T2VEval: Benchmark Dataset and Objective Evaluation Method for T2V-generated Videoscited by 6
  14. 2025AEGIS: Authenticity Evaluation Benchmark for AI-Generated Video Sequencescited by 6
  15. 2025A Granular Study of Safety Pretraining under Model Abliterationcited by 6
  16. 2026VII: Visual Instruction Injection for Jailbreaking Image-to-Video Generation Modelscited by 6
  17. 2026Evolution of Video Generative Foundationscited by 6
  18. 2025Unified Prompt Attack Against Text-to-Image Generation Modelscited by 5
  19. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Datacited by 5
  20. 2026Time Is All It Takes: Spike-Retiming Attacks on Event-Driven Spiking Neural Networkscited by 5
  21. 2026JailWAM: Jailbreaking World Action Models in Robot Controlcited by 5
  22. 2025SPARK: Jailbreaking T2V Models by Synergistically Prompting Auditory and Recontextualized Knowledgecited by 4
  23. 2026VidLeaks: Membership Inference Attacks Against Text-to-Video Modelscited by 4
  24. 2025AIGVE-MACS: Unified Multi-Aspect Commenting and Scoring Model for AI-Generated Video Evaluationcited by 3
  25. 2025Now You See It, Now You Don't - Instant Concept Erasure for Safe Text-to-Image and Video Generationcited by 3
  26. 2025TEAR: Temporal-aware Automated Red-teaming for Text-to-Video Modelscited by 3
  27. 2026Two Frames Matter: A Temporal Attack for Text-to-Video Model Jailbreakingcited by 3
  28. 2026SafeGen-Bench: Benchmarking Safety in Image-Conditioned Text-to-Video Generationcited by 3
  29. 2025A Red Teaming Roadmap Towards System-Level Safetycited by 2
  30. 2026Why Do Unlearnable Examples Work: A Novel Perspective of Mutual Informationcited by 2
  31. 2026Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Controlcited by 2
  32. 2026Do Models Share Safety Representations? Cross-Model Steering for Safe Visual Generationcited by 2
  33. 2025Inference-Time Alignment of Diffusion Models via Evolutionary Algorithmscited by 1
  34. 2025Investigating Vulnerabilities and Defenses Against Audio-Visual Attacks: A Comprehensive Survey Emphasizing Multimodal Modelscited by 1
  35. 2025SeqBench: Benchmarking Sequential Narrative Generation in Text-to-Video Modelscited by 1
  36. 2026AI Security in the Foundation Model Era: A Comprehensive Survey from a Unified Perspectivecited by 1
  37. 2026TrajShield: Trajectory-Level Safety Mediation for Defending Text-to-Video Models Against Jailbreak Attackscited by 1
  38. 2026VPA-Guard: Defending and Benchmarking Image-to-Video Generation Against Visual Prompt Attackscited by 1

50 most cited of 66 All 66 (JSON)