mlsanityData to 14 Oct 2026

Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioning

arXiv 2311.10709 ↗First posted 17 Nov 2023cs.CV cs.AI cs.GRcited by 255

Not on the map. Why?

Cited by 116 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present Emu Video, a text-to-video generation model that factorizes the generation into two steps: first generating an image conditioned on the text, and then generating a video conditioned on the text and the generated image. We identify critical design decisions--adjusted noise schedules for diffusion, and multi-stage training that enable us to directly generate high quality and high resolution videos, without requiring a deep cascade of models as in prior work. In human evaluations, our generated videos are strongly preferred in quality compared to all prior work--81% vs. Google's Imagen Video, 90% vs. Nvidia's PYOCO, and 96% vs. Meta's Make-A-Video. Our model outperforms commercial solutions such as RunwayML's Gen2 and Pika Labs. Finally, our factorizing approach naturally lends itself to animating images based on a user's text prompt, where our generations are preferred 96% over prior work.

Key facts. Cited by 255 papers in the map (data through 14 Oct 2026). Builds on 79 of its 90 references that are in the map. First posted on arXiv 17 Nov 2023.

Builds on

79 of 90 references.

References · 79

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2022Classifier-Free Diffusion Guidancecited by 5,847
  11. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,288
38 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  2. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  4. 2016Improved Techniques for Training GANscited by 2,970
  5. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  6. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  7. 2022Video Diffusion Modelscited by 2,305
  8. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  9. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  10. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  11. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  12. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  13. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  14. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  15. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  16. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 670
  17. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  18. 2023ModelScope Text-to-Video Technical Reportcited by 658
  19. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 590
  20. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 535
  21. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  22. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  23. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  24. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 465
  25. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  26. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 388
  27. 2016Generating Videos with Scene Dynamicscited by 376
  28. 2016Adversarial Feature Learningcited by 350
  29. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 350
  30. 2022MAGVIT: Masked Generative Video Transformercited by 346
  31. 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 327
  32. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 310
  33. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 309
  34. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 307
  35. 2015Deep multi-scale video prediction beyond mean square errorcited by 300
  36. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  37. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 285
  38. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 279

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 9; 2024: 143; 2025: 79; 2026: 24.

255 papers.

Sort:

By area:

  1. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  2. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  3. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  4. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
  5. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
  6. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  7. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 347
  8. 2024Navigation World Modelscited by 272
  9. 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 259
  10. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Textcited by 251
  11. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  12. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 212
38 more
  1. 2025Unified Video Action Modelcited by 205
  2. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 197
  3. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 191
  4. 2024VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Timecited by 187
  5. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 178
  6. 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Modelscited by 175
  7. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 174
  8. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  9. 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Modelscited by 161
  10. 2024Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Modelscited by 160
  11. 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
  12. 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 153
  13. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 146
  14. 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 144
  15. 2024FIFO-Diffusion: Generating Infinite Videos from Text without Trainingcited by 137
  16. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 132
  17. 2024VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Modelscited by 131
  18. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representationcited by 123
  19. 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Modelcited by 120
  20. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 116
  21. 2025VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Modelscited by 113
  22. 2025Long Context Tuning for Video Generationcited by 113
  23. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  24. 2024Adaptive Caching for Faster Video Generation with Diffusion Transformerscited by 108
  25. 2024Boximator: Generating Rich and Controllable Motions for Video Synthesiscited by 107
  26. 2024Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenizationcited by 100
  27. 2024Loong: Generating Minute-level Long Videos with Autoregressive Language Modelscited by 99
  28. 2025Controllable Video Generation: A Surveycited by 98
  29. 2024SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpointscited by 93
  30. 2024DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmarkcited by 92
  31. 2024Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrappingcited by 90
  32. 2025WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generationcited by 90
  33. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokenscited by 89
  34. 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 89
  35. 2024ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restorationcited by 89
  36. 2024IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generationcited by 84
  37. 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 78
  38. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 76

50 most cited of 255 All 255 (JSON)