mlsanityData to 7 Oct 2026

Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models

arXiv 2405.04233 ↗First posted 7 May 2024cs.CV cs.LGcited by 159

On the map: Diffusion models → Video generation

Abstract (arXiv)

We introduce Vidu, a high-performance text-to-video generator that is capable of producing 1080p videos up to 16 seconds in a single generation. Vidu is a diffusion model with U-ViT as its backbone, which unlocks the scalability and the capability for handling long videos. Vidu exhibits strong coherence and dynamism, and is capable of generating both realistic and imaginative videos, as well as understanding some professional photography techniques, on par with Sora -- the most powerful reported text-to-video generator. Finally, we perform initial experiments on other controllable video generation, including canny-to-video generation, video prediction and subject-driven generation, which demonstrate promising results.

Key facts. Cited by 159 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 13 of its 16 references that are in the map. First posted on arXiv 7 May 2024.

Builds on

13 of 16 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  2. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  3. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct

Other references · 10

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  8. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  9. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  10. 2023One Transformer Fits All Distributions in Multi-Modal Diffusion at Scalecited by 185

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 38; 2025: 71; 2026: 50.

159 papers.

Sort:

By area:

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  2. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  3. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  4. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  5. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  6. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 204
  7. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
  8. 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generationcited by 161
  9. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 159
  10. 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
  11. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 155
  12. 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 121
38 more
  1. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 120
  2. 2025SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inferencecited by 113
  3. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 111
  4. 2024IRASim: A Fine-Grained World Model for Robot Manipulationcited by 93
  5. 2024FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attentioncited by 87
  6. 2024StableAnimator: High-Quality Identity-Preserving Human Image Animationcited by 85
  7. 2024Pandora: Towards General World Model with Natural Language Actions and Video Statescited by 83
  8. 2025Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistencycited by 81
  9. 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
  10. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  11. 2025Goku: Flow Based Video Generative Foundation Modelscited by 67
  12. 2024T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Modelscited by 66
  13. 2025RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformerscited by 63
  14. 2025RewardDance: Reward Scaling in Visual Generationcited by 63
  15. 2025EnerVerse: Envisioning Embodied Future Space for Robotics Manipulationcited by 59
  16. 2025Vidar: Embodied Video Diffusion Model for Generalist Manipulationcited by 58
  17. 2025STAR: Spatial-Temporal Augmentation with Text-to-Video Models for Real-World Video Super-Resolutioncited by 56
  18. 2024WF-VAE: Enhancing Video VAE by Wavelet-Driven Energy Flow for Latent Video Diffusion Modelcited by 55
  19. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 54
  20. 2026Motubrain: An Advanced World Action Model for Robot Controlcited by 54
  21. 2024Diffusion Bridge Implicit Modelscited by 47
  22. 2025EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Modelscited by 45
  23. 2025Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidancecited by 45
  24. 2025SnapMoGen: Human Motion Generation from Expressive Textscited by 44
  25. 2026Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generationcited by 44
  26. 2025StableAvatar: Infinite-Length Audio-Driven Avatar Video Generationcited by 41
  27. 2024Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfelscited by 38
  28. 2024OnlineVPO: Align Video Diffusion Model with Online Video-Centric Preference Optimizationcited by 38
  29. 2025VideoShield: Regulating Diffusion-based Video Generation Models via Watermarkingcited by 32
  30. 2025FlashVideo: Flowing Fidelity to Detail for Efficient High-Resolution Video Generationcited by 32
  31. 2024OD-VAE: An Omni-dimensional Video Compressor for Improving Latent Video Diffusion Modelcited by 30
  32. 2024The Dawn of Video Generation: Preliminary Explorations with SORA-like Modelscited by 30
  33. 2025MotionPro: A Precise Motion Controller for Image-to-Video Generationcited by 29
  34. 2024Identifying and Solving Conditional Image Leakage in Image-to-Video Diffusion Modelcited by 27
  35. 2025Omni-Effects: Unified and Spatially-Controllable Visual Effects Generationcited by 27
  36. 2025A Survey of Interactive Generative Videocited by 23
  37. 2025AnyPos: Automated Task-Agnostic Actions for Bimanual Manipulationcited by 23
  38. 2026Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Modelscited by 23

50 most cited of 159 All 159 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation
  2. 2024Grid Diffusion Models for Text-to-Video Generation
  3. 2025VidCLearn: A Continual Learning Approach for Text-to-Video Generation
  4. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators
  5. 2023MEVG: Multi-event Video Generation with Text-to-Video Models
  6. 2022Video Diffusion Models
  7. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text
  8. 2024STIV: Scalable Text and Image Conditioned Video Generation