mlsanityData to 7 Oct 2026

Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Control

arXiv 2606.04775 ↗First posted 3 Jun 2026cs.LG cs.AI cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Text-to-video (T2V) models trained on large-scale web data can generate undesired content, motivating interventions that reduce harmful outputs without sacrificing visual quality. Activation steering offers an attractive mechanistic alternative to finetuning and prompt filtering, but existing T2V steering methods remain limited, typically applying coarse, non-anticipative interventions that can lead to oversteering and content degradation. To close this gap, we propose Latent Activation Linear-Quadratic Regulator (LA-LQR), a reduced-order optimal control framework for minimally invasive T2V steering. LA-LQR formulates T2V inference as a dynamical system and computes closed-loop feedback interventions that steer activations toward desired feature setpoints while penalizing unnecessary perturbations. To make optimal control feasible for high-dimensional video activations, we project activations onto a low-dimensional, task-relevant subspace derived from contrastive prompt pairs, estimate local linear dynamics in this latent space, and solve a latent LQR problem to obtain timestep- and layer-specific steering signals. We provide theoretical bounds relating latent setpoint tracking to raw activation-space feature control, and empirically validate the fidelity of the reduced latent dynamics. On concept steering and video safety benchmarks, LA-LQR reduces unsafe generations relative to baselines, while preserving prompt fidelity and visual quality.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 66 of its 75 references that are in the map. First posted on arXiv 3 Jun 2026.

Builds on

66 of 75 references.

References · 66

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  6. 2020Learning to summarize from human feedbackcited by 2,562
  7. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  8. 2022Video Diffusion Modelscited by 2,280
  9. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  10. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  11. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  12. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
38 more
  1. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  2. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  3. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  4. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  5. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  6. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  7. 2022Toy Models of Superpositioncited by 928
  8. 2023Steering Language Models With Activation Engineeringcited by 824
  9. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  10. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  11. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  12. 2023ModelScope Text-to-Video Technical Reportcited by 651
  13. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
  14. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  15. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
  16. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  17. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 390
  18. 2023Unified Concept Editing in Diffusion Modelscited by 377
  19. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  20. 2023Preference Ranking Optimization for Human Alignmentcited by 283
  21. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  22. 2024ReFT: Representation Finetuning for Language Modelscited by 186
  23. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 156
  24. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 130
  25. 2024SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generationcited by 125
  26. 2024ARGS: Alignment as Reward-Guided Searchcited by 99
  27. 2024T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Modelscited by 66
  28. 2024Aligning Large Language Models with Representation Editing: A Control Perspectivecited by 62
  29. 2024Controlling Language and Diffusion Models by Transporting Activationscited by 59
  30. 2024DeAL: Decoding-time Alignment for Large Language Modelscited by 55
  31. 2025EditVerse: Unifying Image and Video Editing and Generation with In-Context Learningcited by 53
  32. 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 45
  33. 2025Pre-Trained Video Generative Models as World Simulatorscited by 43
  34. 2023What's the Magic Word? A Control Theory of LLM Promptingcited by 41
  35. 2024Advancing Parameter Efficiency in Fine-tuning via Representation Editingcited by 37
  36. 2024Multi-property Steering of Large Language Models with Dynamic Activation Compositioncited by 35
  37. 2024MovieBench: A Hierarchical Movie Level Dataset for Long Video Generationcited by 35
  38. 2024SafeSora: Towards Safety Alignment of Text2Video Generation via a Human Preference Datasetcited by 34

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.