mlsanityData to 7 Oct 2026

Video Generation From Text

arXiv 1710.00421 ↗First posted 1 Oct 2017cs.MMcited by 153

Not on the map. Why?

Cited by 36 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Generating videos from text has proven to be a significant challenge for existing generative models. We tackle this problem by training a conditional generative model to extract both static and dynamic information from text. This is manifested in a hybrid framework, employing a Variational Autoencoder (VAE) and a Generative Adversarial Network (GAN). The static features, called "gist," are used to sketch text-conditioned background color and object layout structure. Dynamic features are considered by transforming input text into an image filter. To obtain a large amount of data for training the deep-learning model, we develop a method to automatically create a matched text-video corpus from publicly available online videos. Experimental results show that the proposed framework generates plausible and diverse videos, while accurately reflecting the input text information. It significantly outperforms baseline models that directly adapt text-to-image generation procedures to produce videos. Performance is evaluated both visually and by adapting the inception score used to evaluate image generation in GANs.

Key facts. Cited by 153 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 23; 2023: 44; 2024: 39; 2025: 22; 2026: 7.

153 papers.

Sort:

By area:

  1. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  2. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  3. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  4. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  5. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 357
  6. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 254
  7. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  8. 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 246
  9. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  10. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 180
  11. 2023NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generationcited by 170
  12. 2023Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generationcited by 153
38 more
  1. 2023Make Pixels Dance: High-Dynamic Video Generationcited by 140
  2. 2023VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planningcited by 138
  3. 2023Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generationcited by 137
  4. 2023Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoisingcited by 135
  5. 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 127
  6. 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
  7. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 112
  8. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  9. 2024Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesiscited by 105
  10. 2023CelebV-Text: A Large-Scale Facial Text-Video Datasetcited by 103
  11. 2024Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependencycited by 102
  12. 2023SimDA: Simple Diffusion Adapter for Efficient Video Generationcited by 101
  13. 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animationcited by 101
  14. 2021Make It Move: Controllable Image-to-Video Generation with Text Descriptionscited by 88
  15. 2024FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attentioncited by 87
  16. 2023InstructVideo: Instructing Video Diffusion Models with Human Feedbackcited by 85
  17. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  18. 2022Make-A-Story: Visual Memory Conditioned Consistent Story Generationcited by 76
  19. 2023VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generationcited by 74
  20. 2023ARTboldsymbol·V: Auto-Regressive Text-to-Video Generation with Diffusion Modelscited by 74
  21. 2024Vlogger: Make Your Dream A Vlogcited by 70
  22. 2024Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Modelcited by 64
  23. 2025HunyuanVideo-Avatar: High-Fidelity Audio-Driven Human Animation for Multiple Characterscited by 64
  24. 2021Predicting Video with VQVAEcited by 61
  25. 2023AI-Generated Content (AIGC): A Surveycited by 59
  26. 2021Speech2AffectiveGestures: Synthesizing Co-Speech Gestures with Generative Adversarial Affective Expression Learningcited by 58
  27. 2024Magic-Me: Identity-Specific Video Customized Diffusioncited by 58
  28. 2021In&Out : Diverse Image Outpainting via GAN Inversioncited by 55
  29. 2024Customize-A-Video: One-Shot Motion Customization of Text-to-Video Diffusion Modelscited by 55
  30. 2021Adversarial Text-to-Image Synthesis: A Reviewcited by 52
  31. 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generationcited by 48
  32. 2023Breathing Life Into Sketches Using Text-to-Video Priorscited by 47
  33. 2021Improving Generation and Evaluation of Visual Stories via Semantic Consistencycited by 43
  34. 2023VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Viewscited by 38
  35. 2022Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioningcited by 37
  36. 2023GPT4Video: A Unified Multimodal Large Language Model for lnstruction-Followed Understanding and Safety-Aware Generationcited by 36
  37. 2023Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMscited by 35
  38. 2025T2VPhysBench: A First-Principles Benchmark for Physical Consistency in Text-to-Video Generationcited by 35

50 most cited of 153 All 153 (JSON)