mlsanityData to 7 Oct 2026

Text-To-4D Dynamic Scene Generation

arXiv 2301.11280 ↗First posted 26 Jan 2023cs.CV cs.AI cs.LGcited by 204

Not on the map. Why?

Cited by 75 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present MAV3D (Make-A-Video3D), a method for generating three-dimensional dynamic scenes from text descriptions. Our approach uses a 4D dynamic Neural Radiance Field (NeRF), which is optimized for scene appearance, density, and motion consistency by querying a Text-to-Video (T2V) diffusion-based model. The dynamic video output generated from the provided text can be viewed from any camera location and angle, and can be composited into any 3D environment. MAV3D does not require any 3D or 4D data and the T2V model is trained only on Text-Image pairs and unlabeled videos. We demonstrate the effectiveness of our approach using comprehensive quantitative and qualitative experiments and show an improvement over previously established internal baselines. To the best of our knowledge, our method is the first to generate 3D dynamic scenes given a text description.

Key facts. Cited by 204 papers in the map (data through 7 Oct 2026). Builds on 44 of its 59 references that are in the map. First posted on arXiv 26 Jan 2023.

Builds on

44 of 59 references.

References · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  7. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  10. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  11. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  12. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
32 more
  1. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  2. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  3. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  4. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  5. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  6. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  7. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  8. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  9. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  10. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  11. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  12. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  13. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 742
  14. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  15. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  16. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  17. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 582
  18. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  19. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  20. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  21. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 470
  22. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  23. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  24. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  25. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  26. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  27. 2022Fast Dynamic Radiance Fields with Time-Aware Neural Voxelscited by 344
  28. 2020Neural Radiance Flow for 4D View Synthesis and Video Processingcited by 270
  29. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  30. 2018Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddingscited by 169
  31. 2022KNN-Diffusion: Image Generation via Large-Scale Retrievalcited by 135
  32. 2021ClipMatrix: Text-controlled Creation of 3D Textured Meshescited by 40

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 41; 2024: 69; 2025: 66; 2026: 28.

204 papers.

Sort:

By area:

  1. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  2. 2023Adversarial Diffusion Distillationcited by 741
  3. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 444
  4. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 296
  5. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  6. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 191
  7. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 185
  8. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 184
  9. 2025World Simulation with Video Foundation Models for Physical AIcited by 183
  10. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  11. 2023DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusioncited by 167
  12. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
38 more
  1. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  2. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  3. 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
  4. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  5. 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 143
  6. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 143
  7. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
  8. 2023Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance Fieldscited by 142
  9. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  10. 2023Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Videocited by 124
  11. 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
  12. 2024Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesiscited by 116
  13. 2023Animate124: Animating One Image to 4D Dynamic Scenecited by 113
  14. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
  15. 2023DreamHuman: Animatable 3D Avatars from Textcited by 102
  16. 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussianscited by 102
  17. 2024GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creationcited by 99
  18. 2024Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Modelscited by 99
  19. 2025TesserAct: Learning 4D Embodied World Modelscited by 90
  20. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  21. 2024EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generationcited by 84
  22. 2023A Unified Approach for Text- and Image-guided 4D Scene Generationcited by 82
  23. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
  24. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priorscited by 72
  25. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 68
  26. 20244Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Modelscited by 64
  27. 2024DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videoscited by 63
  28. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  29. 2024SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfercited by 61
  30. 2024DreamMesh4D: Video-to-4D Generation with Sparse-Controlled Gaussian-Mesh Hybrid Representationcited by 58
  31. 2024Animate3D: Animating Any 3D Model with Multi-view Video Diffusioncited by 55
  32. 2024Comp4D: LLM-Guided Compositional 4D Scene Generationcited by 50
  33. 2024GenXD: Generating Any 3D and 4D Scenescited by 49
  34. 2023Breathing Life Into Sketches Using Text-to-Video Priorscited by 47
  35. 2025RigAnything: Template-Free Autoregressive Rigging for Diverse 3D Assetscited by 47
  36. 2023A Comprehensive Survey on Knowledge Distillation of Diffusion Modelscited by 45
  37. 2024EG4D: Explicit Generation of 4D Object without Score Distillationcited by 45
  38. 2024Efficient4D: Fast Dynamic 3D Object Generation from a Single-view Videocited by 44

50 most cited of 204 All 204 (JSON)