Dimension-Reduction Attack! Video Generative Models are Experts on Controllable Image Synthesis
Not on the map. Why?
Abstract (arXiv)
Video generative models can be regarded as world simulators due to their ability to capture dynamic, continuous changes inherent in real-world environments. These models integrate high-dimensional information across visual, temporal, spatial, and causal dimensions, enabling predictions of subjects in various status. A natural and valuable research direction is to explore whether a fully trained video generative model in high-dimensional space can effectively support lower-dimensional tasks such as controllable image generation. In this work, we propose a paradigm for video-to-image knowledge compression and task adaptation, termed textitDimension-Reduction Attack (textttDRA-Ctrl), which utilizes the strengths of video models, including long-range context modeling and flatten full-attention, to perform various generation tasks. Specially, to address the challenging gap between continuous video frames and discrete image generation, we introduce a mixup-based transition strategy that ensures smooth adaptation. Moreover, we redesign the attention structure with a tailored masking mechanism to better align text prompts with image-level control. Experiments across diverse image generation tasks, such as subject-driven and spatially conditioned generation, show that repurposed video models outperform those trained directly on images. These results highlight the untapped potential of large-scale video generators for broader visual applications. textttDRA-Ctrl provides new insights into reusing resource-intensive video models and lays foundation for future unified generative models across visual modalities. The project page is https://dra-ctrl-2025.github.io/DRA-Ctrl/.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 45 of its 64 references that are in the map. First posted on arXiv 29 May 2025.
Builds on
45 of 64 references.
References · 45
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
33 more
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
- 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
- 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
- 2020Pre-Trained Image Processing Transformercited by 893
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
- 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 688
- 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
- 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 412
- 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Modelscited by 359
- 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
- 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 235
- 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 210
- 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wildcited by 200
- 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 154
- 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 144
- 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 143
- 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 119
- 2025ACE++: Instruction-Based Image Creation and Editing via Context-Aware Content Fillingcited by 95
- 2023Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Followingcited by 77
- 2024ACE: All-round Creator and Editor Following Instructions via Diffusion Transformercited by 56
- 2024JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generationcited by 52
- 2023DreamTuner: Single Image is Enough for Subject-Driven Generationcited by 41
- 2025VisualCloze: A Universal Image Generation Framework via Visual In-Context Learningcited by 39
- 2025ObjectMover: Generative Object Movement with Video Priorcited by 30
- 2024BootPIG: Bootstrapping Zero-shot Personalized Image Generation Capabilities in Pretrained Diffusion Modelscited by 27
- 2025Generating Multi-Image Synthetic Data for Text-to-Image Customizationcited by 25
- 2024OutfitAnyone: Ultra-high Quality Virtual Try-On for Any Clothing and Any Personcited by 16
- 2025RealGeneral: Unifying Visual Generation via Temporal In-Context Learning with Video Modelscited by 16
- 2025FramePainter: Endowing Interactive Image Editing with Video Diffusion Priorscited by 13
- 2025FlexIP: Dynamic Control of Preservation and Personality for Customized Image Generationcited by 3
Built on by
Citing papers per year: 2025: 3; 2026: 3.
6 papers.
- 2025Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillationcited by 86
- 2025Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferencescited by 13
- 2026DMGD: Train-Free Dataset Distillation with Semantic-Distribution Matching in Diffusion Modelscited by 3
- 2025DreamingComics: A Story Visualization Pipeline via Subject and Layout Customized Generation using Video Modelscited by 1
- 2026Offline Preference Optimization for Rectified Flow with Noise-Tracked Pairscited by 1
- 2026Chamaileon: Cross-Context Binder Design with Contextualized Modeling and Mixed Samplingcited by 0
- 2026Chamaileon: Cross-Context Binder Design with Contextualized Modeling and Mixed Sampling26 Jul 2026
- 2026Offline Preference Optimization for Rectified Flow with Noise-Tracked Pairs10 May 2026
- 2026DMGD: Train-Free Dataset Distillation with Semantic-Distribution Matching in Diffusion Models5 May 2026
- 2025Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillation4 Dec 2025
- 2025DreamingComics: A Story Visualization Pipeline via Subject and Layout Customized Generation using Video Models1 Dec 2025
- 2025Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferences3 Jun 2025