Video Unlearning via Low-Rank Refusal Vector
On the map: LLM safety and security → Vision-language and text-to-image safety
Abstract (arXiv)
Video generative models achieve high-quality synthesis from natural-language prompts by leveraging large-scale web data. However, this training paradigm inherently exposes them to unsafe biases and harmful concepts, introducing the risk of generating undesirable or illicit content. To mitigate unsafe generations, existing machine unlearning approaches either rely on filtering, and can therefore be bypassed, or they update model weights, but with costly fine-tuning or training-free closed-form edits. We propose the first training-free weight update framework for concept removal in video diffusion models. From five paired safe/unsafe prompts, our method estimates a refusal vector and integrates it into the model weights as a closed-form update. A contrastive low-rank factorization further disentangles the target concept from unrelated semantics, it ensures a selective concept suppression and it does not harm generation quality. Our approach reduces unsafe generations on the Open-Sora and ZeroScopeT2V models across the T2VSafetyBench and SafeSora benchmarks, with average reductions of 36.3% and 58.2% respectively, while preserving prompt alignment and video quality. This establishes an efficient and scalable solution for safe video generation without retraining nor any inference overhead. Project page: https://www.pinlab.org/video-unlearning.
Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 43 of its 56 references that are in the map. First posted on arXiv 9 Jun 2025.
Builds on
43 of 56 references.
Closest ancestors · 6
Same direction, newest first
- 2024SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generation16 Oct
- 2024Unstable Unlearning: The Hidden Risk of Concept Resurgence in Diffusion Models10 Oct
- 2024SteerDiff: Steering towards Safe Text-to-Image Diffusion Models3 Oct
- 2024Unlearning Concepts from Text-to-Video Diffusion Models19 Jul
- 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now18 Oct
- 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models30 Mar
Other references · 37
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2020Learning to summarize from human feedbackcited by 2,588
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
25 more
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
- 2022Toy Models of Superpositioncited by 971
- 2023Steering Language Models With Activation Engineeringcited by 922
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
- 2023ModelScope Text-to-Video Technical Reportcited by 658
- 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 652
- 2023Erasing Concepts from Diffusion Modelscited by 539
- 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 513
- 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
- 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
- 2023Unified Concept Editing in Diffusion Modelscited by 388
- 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
- 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
- 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 307
- 2025Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200kcited by 163
- 2024The Geometry of Categorical and Hierarchical Concepts in Large Language Modelscited by 159
- 2024Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Modelscited by 133
- 2023Fine-tuning can cripple your foundation model; preserving features may be the solutioncited by 96
- 2024Controlling Language and Diffusion Models by Transporting Activationscited by 68
- 2024T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Modelscited by 66
- 2024Fr'echet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videoscited by 58
- 2024SafeSora: Towards Safety Alignment of Text2Video Generation via a Human Preference Datasetcited by 35
- 2022The Stable Artist: Steering Semantics in Diffusion Latent Spacecited by 23
Built on by
Citing papers per year: 2026: 13.
13 papers.
Sort:
By area:
- 2026Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarkscited by 7
- 2026Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Controlcited by 3
- 2026Steering Robustness into World Action Models via Mechanistic Interpretability and Optimal Controlcited by 3
- 2026Do Models Share Safety Representations? Cross-Model Steering for Safe Visual Generationcited by 2
- 2026VPA-Guard: Defending and Benchmarking Image-to-Video Generation Against Visual Prompt Attackscited by 1
- 2026FADE: Frame-Aware Diffusion-Transformer-based Multi-Concept Erasure for Video Unlearningcited by 1
- 2026Your Unlearning Gives You Away: Identifying Erased Concepts in Diffusion Modelscited by 1
- 2026How to Take a Memorable Picture? Empowering Users with Actionable Feedbackcited by 0
- 2026SoK: Unlearnability and Unlearning for Model Dememorizationcited by 0
- 2026Inference-Time Concept Suppression and Video-Centric Evaluation for Text-to-Video Modelscited by 0
- 2026Motion Concept Unlearning in Video Diffusion Modelscited by 0
- 2026Steering Fields: Adaptive Vector Fields for Safe Image Generation and Beyondcited by 0
- 2026Your Unlearning Gives You Away: Identifying Erased Concepts in Diffusion Models4 Oct 2026
- 2026FADE: Frame-Aware Diffusion-Transformer-based Multi-Concept Erasure for Video Unlearning2 Oct 2026
- 2026FOMO: Forget the Concept, Don't Miss Out on the Scene in Selective Video Unlearning30 Sep 2026
- 2026Steering Fields: Adaptive Vector Fields for Safe Image Generation and Beyond30 Sep 2026
- 2026Motion Concept Unlearning in Video Diffusion Models29 Sep 2026
- 2026Steering Robustness into World Action Models via Mechanistic Interpretability and Optimal Control16 Jul 2026
- 2026Inference-Time Concept Suppression and Video-Centric Evaluation for Text-to-Video Models15 Jul 2026
- 2026Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks8 Jul 2026
- 2026VPA-Guard: Defending and Benchmarking Image-to-Video Generation Against Visual Prompt Attacks24 Jun 2026
- 2026Do Models Share Safety Representations? Cross-Model Steering for Safe Visual Generation3 Jun 2026
- 2026Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Control3 Jun 2026
- 2026SoK: Unlearnability and Unlearning for Model Dememorization12 May 2026
1 more
- 2026Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarkscited by 7
- 2026SoK: Unlearnability and Unlearning for Model Dememorizationcited by 0
- 2026Inference-Time Concept Suppression and Video-Centric Evaluation for Text-to-Video Modelscited by 0
- 2026Steering Fields: Adaptive Vector Fields for Safe Image Generation and Beyondcited by 0
- 2026FOMO: Forget the Concept, Don't Miss Out on the Scene in Selective Video Unlearningcited by 0
- 2026FOMO: Forget the Concept, Don't Miss Out on the Scene in Selective Video Unlearning30 Sep 2026
- 2026Steering Fields: Adaptive Vector Fields for Safe Image Generation and Beyond30 Sep 2026
- 2026Inference-Time Concept Suppression and Video-Centric Evaluation for Text-to-Video Models15 Jul 2026
- 2026Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks8 Jul 2026
- 2026SoK: Unlearnability and Unlearning for Model Dememorization12 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025T2VUnlearning: A Concept Erasing Method for Text-to-Video Diffusion Models
- 2025VideoEraser: Concept Erasure in Text-to-Video Diffusion Models
- 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models
- 2025Pre-Forgettable Models: Prompt Learning as a Native Mechanism for Unlearning
- 2026Why Instruction-Based Unlearning Fails in Diffusion Models?
- 2026Unlearning the Unpromptable: Prompt-free Instance Unlearning in Diffusion Models
- 2024ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruning
- 2025Prompt-Based Safety Guidance Is Ineffective for Unlearned Text-to-Image Diffusion Models