mlsanityData to 7 Oct 2026

UniCtrl: Improving the Spatiotemporal Consistency of Text-to-Video Diffusion Models via Training-Free Unified Attention Control

arXiv 2403.02332 ↗First posted 4 Mar 2024cs.CVcited by 14

On the map: Diffusion models → Video generation

Abstract (arXiv)

Video Diffusion Models have been developed for video generation, usually integrating text and image conditioning to enhance control over the generated content. Despite the progress, ensuring consistency across frames remains a challenge, particularly when using text prompts as control conditions. To address this problem, we introduce UniCtrl, a novel, plug-and-play method that is universally applicable to improve the spatiotemporal consistency and motion diversity of videos generated by text-to-video models without additional training. UniCtrl ensures semantic consistency across different frames through cross-frame self-attention control, and meanwhile, enhances the motion quality and spatiotemporal consistency through motion injection and spatiotemporal synchronization. Our experimental results demonstrate UniCtrl's efficacy in enhancing various text-to-video models, confirming its effectiveness and universality.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 67 of its 82 references that are in the map. First posted on arXiv 4 Mar 2024.

Builds on

67 of 82 references.

Closest ancestors · 19

Same direction, newest first

  1. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models17 Jan
  2. 2023FreeInit: Bridging Initialization Gap in Video Diffusion Models12 Dec
  3. 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models28 Nov
  4. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  5. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generation30 Oct
  6. 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling23 Oct
  7. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors18 Oct
  8. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation27 Sep
  9. 2023Reuse and Diffuse: Iterative Denoising for Text-to-Video Generation7 Sep
  10. 2023ModelScope Text-to-Video Technical Report12 Aug
  11. 2023VideoControlNet: A Motion-Guided Video-to-Video Translation Framework by Using Diffusion Model with ControlNet26 Jul
  12. 2023InFusion: Inject and Attention Fusion for Multi Concept Zero-Shot Text-based Video Editing22 Jul
7 more
  1. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  2. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation13 Jun
  3. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models17 May
  4. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  5. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators23 Mar
  6. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct
  7. 2022Video Diffusion Models7 Apr

Other references · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  11. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
36 more
  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  2. 2022Scalable Diffusion Models with Transformerscited by 5,931
  3. 2022Generative Adversarial Networkscited by 5,603
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2017Neural Discrete Representation Learningcited by 4,650
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  8. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  10. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  11. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  12. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  13. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  14. 2023Consistency Modelscited by 1,778
  15. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  16. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  17. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  18. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  19. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
  20. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
  21. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  22. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  23. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  24. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  25. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 254
  26. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  27. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  28. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 162
  29. 2022Generating Long Videos of Dynamic Scenescited by 113
  30. 2023Inversion-Free Image Editing with Natural Languagecited by 106
  31. 2023Expressive Text-to-Image Generation with Rich Textcited by 89
  32. 2021Make It Move: Controllable Image-to-Video Generation with Text Descriptionscited by 88
  33. 2023Seer: Language Instructed Video Prediction with Latent Diffusion Modelscited by 51
  34. 2023CycleNet: Rethinking Cycle Consistency in Text-Guided Diffusion for Image Manipulationcited by 41
  35. 2023PIA: Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Modelscited by 30
  36. 2023Style-A-Video: Agile Diffusion for Arbitrary Text-based Video Style Transfercited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 3; 2026: 3.

14 papers.

Sort:

By area:

  1. 2024Sora as a World Model? A Complete Survey on Text-to-Video Generationcited by 53
  2. 2024Optical-Flow Guided Prompt Optimization for Coherent Video Generationcited by 17
  3. 2024I4VGen: Image as Free Stepping Stone for Text-to-Video Generationcited by 12
  4. 2024CTRL-D: Controllable Dynamic 3D Scene Editing with Personalized 2D Diffusioncited by 11
  5. 2024IV-Mixed Sampler: Leveraging Image Diffusion Models for Enhanced Video Synthesiscited by 8
  6. 2024ByTheWay: Boost Your Text-to-Video Generation Model to Higher Quality in a Training-free Waycited by 7
  7. 2025DropletVideo: A Dataset and Approach to Explore Integral Spatio-Temporal Consistent Video Generationcited by 6
  8. 2024VideoGuide: Improving Video Diffusion Models without Training Through a Teacher's Guidecited by 5
  9. 2026PhysVid: Physics Aware Local Conditioning for Generative Video Modelscited by 5
  10. 2024RefDrop: Controllable Consistency in Image or Video Generation via Reference Feature Guidancecited by 4
  11. 2025V.I.P. : Iterative Online Preference Distillation for Efficient Video Diffusion Modelscited by 1
  12. 2025Video Consistency Distance: Enhancing Temporal Consistency for Image-to-Video Generation via Reward-Based Fine-Tuningcited by 1
2 more
  1. 2026Steering Video Diffusion Transformers with Massive Activationscited by 0
  2. 2026Embedding Prediction Helps Image Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026UniVid: Pyramid Diffusion Model for High Quality Video Generation
  2. 2025Enabling Versatile Controls for Video Diffusion Models
  3. 2025TempoControl: Temporal Attention Guidance for Text-to-Video Models
  4. 2023ControlVideo: Training-free Controllable Text-to-Video Generation
  5. 2024DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation
  6. 2025InstanceV: Instance-Level Video Generation
  7. 2023LLM-grounded Video Diffusion Models
  8. 2025EIDT-V: Exploiting Intersections in Diffusion Trajectories for Model-Agnostic, Zero-Shot, Training-Free Text-to-Video Generation