Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Control
Not on the map. Why?
Abstract (arXiv)
Text-to-video (T2V) models trained on large-scale web data can generate undesired content, motivating interventions that reduce harmful outputs without sacrificing visual quality. Activation steering offers an attractive mechanistic alternative to finetuning and prompt filtering, but existing T2V steering methods remain limited, typically applying coarse, non-anticipative interventions that can lead to oversteering and content degradation. To close this gap, we propose Latent Activation Linear-Quadratic Regulator (LA-LQR), a reduced-order optimal control framework for minimally invasive T2V steering. LA-LQR formulates T2V inference as a dynamical system and computes closed-loop feedback interventions that steer activations toward desired feature setpoints while penalizing unnecessary perturbations. To make optimal control feasible for high-dimensional video activations, we project activations onto a low-dimensional, task-relevant subspace derived from contrastive prompt pairs, estimate local linear dynamics in this latent space, and solve a latent LQR problem to obtain timestep- and layer-specific steering signals. We provide theoretical bounds relating latent setpoint tracking to raw activation-space feature control, and empirically validate the fidelity of the reduced latent dynamics. On concept steering and video safety benchmarks, LA-LQR reduces unsafe generations relative to baselines, while preserving prompt fidelity and visual quality.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 66 of its 75 references that are in the map. First posted on arXiv 3 Jun 2026.
Builds on
66 of 75 references.
References · 66
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2020Learning to summarize from human feedbackcited by 2,562
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2022Video Diffusion Modelscited by 2,280
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
38 more
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
- 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
- 2022Toy Models of Superpositioncited by 928
- 2023Steering Language Models With Activation Engineeringcited by 824
- 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
- 2023ModelScope Text-to-Video Technical Reportcited by 651
- 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
- 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
- 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
- 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
- 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 390
- 2023Unified Concept Editing in Diffusion Modelscited by 377
- 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
- 2023Preference Ranking Optimization for Human Alignmentcited by 283
- 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
- 2024ReFT: Representation Finetuning for Language Modelscited by 186
- 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 156
- 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 130
- 2024SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generationcited by 125
- 2024ARGS: Alignment as Reward-Guided Searchcited by 99
- 2024T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Modelscited by 66
- 2024Aligning Large Language Models with Representation Editing: A Control Perspectivecited by 62
- 2024Controlling Language and Diffusion Models by Transporting Activationscited by 59
- 2024DeAL: Decoding-time Alignment for Large Language Modelscited by 55
- 2025EditVerse: Unifying Image and Video Editing and Generation with In-Context Learningcited by 53
- 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 45
- 2025Pre-Trained Video Generative Models as World Simulatorscited by 43
- 2023What's the Magic Word? A Control Theory of LLM Promptingcited by 41
- 2024Advancing Parameter Efficiency in Fine-tuning via Representation Editingcited by 37
- 2024Multi-property Steering of Large Language Models with Dynamic Activation Compositioncited by 35
- 2024MovieBench: A Hierarchical Movie Level Dataset for Long Video Generationcited by 35
- 2024SafeSora: Towards Safety Alignment of Text2Video Generation via a Human Preference Datasetcited by 34
50 most cited of 66 All 66 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.