mlsanityData to 7 Oct 2026

Animate124: Animating One Image to 4D Dynamic Scene

arXiv 2311.14603 ↗First posted 24 Nov 2023cs.CVcited by 113

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We introduce Animate124 (Animate-one-image-to-4D), the first work to animate a single in-the-wild image into 3D video through textual motion descriptions, an underexplored problem with significant applications. Our 4D generation leverages an advanced 4D grid dynamic Neural Radiance Field (NeRF) model, optimized in three distinct stages using multiple diffusion priors. Initially, a static model is optimized using the reference image, guided by 2D and 3D diffusion priors, which serves as the initialization for the dynamic NeRF. Subsequently, a video diffusion model is employed to learn the motion specific to the subject. However, the object in the 3D videos tends to drift away from the reference image over time. This drift is mainly due to the misalignment between the text prompt and the reference image in the video diffusion model. In the final stage, a personalized diffusion prior is therefore utilized to address the semantic drift. As the pioneering image-text-to-4D generation framework, our method demonstrates significant advancements over existing baselines, evidenced by comprehensive quantitative and qualitative assessments.

Key facts. Cited by 113 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 44 of its 51 references that are in the map. First posted on arXiv 24 Nov 2023.

Builds on

44 of 51 references.

Closest ancestors · 11

Same direction, newest first

  1. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior25 Oct
  2. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  3. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
  4. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
  5. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  6. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  7. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  8. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  9. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views29 Nov
  10. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  11. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  8. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  9. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  10. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  11. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  12. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
21 more
  1. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  2. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  3. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 742
  4. 2023ModelScope Text-to-Video Technical Reportcited by 651
  5. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  6. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  7. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 582
  8. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  9. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  10. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  11. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  12. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  13. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  14. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  15. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
  16. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  17. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  18. 2022Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Renderingcited by 217
  19. 2023Text-To-4D Dynamic Scene Generationcited by 204
  20. 2023Temporal Interpolation Is All You Need for Dynamic Neural Radiance Fieldscited by 59
  21. 2022Understanding Pure CLIP Guidance for Voxel Grid NeRF Modelscited by 56

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 44; 2025: 47; 2026: 17.

113 papers.

Sort:

By area:

  1. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 191
  2. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 184
  3. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  4. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  5. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  6. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  7. 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 143
  8. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 143
  9. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
  10. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 139
  11. 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
  12. 2024Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesiscited by 116
38 more
  1. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
  2. 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussianscited by 102
  3. 2024GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creationcited by 99
  4. 2024Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Modelscited by 99
  5. 2025TesserAct: Learning 4D Embodied World Modelscited by 90
  6. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
  7. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 68
  8. 2024DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videoscited by 63
  9. 2024SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfercited by 61
  10. 2024DreamMesh4D: Video-to-4D Generation with Sparse-Controlled Gaussian-Mesh Hybrid Representationcited by 58
  11. 2024Animate3D: Animating Any 3D Model with Multi-view Video Diffusioncited by 55
  12. 20244K4DGen: Panoramic 4D Generation at 4K Resolutioncited by 54
  13. 2024Comp4D: LLM-Guided Compositional 4D Scene Generationcited by 50
  14. 2024GenXD: Generating Any 3D and 4D Scenescited by 49
  15. 2025RigAnything: Template-Free Autoregressive Rigging for Diverse 3D Assetscited by 47
  16. 2024EG4D: Explicit Generation of 4D Object without Score Distillationcited by 45
  17. 2024Efficient4D: Fast Dynamic 3D Object Generation from a Single-view Videocited by 44
  18. 2025SV4D 2.0: Enhancing Spatio-Temporal Consistency in Multi-View Video Diffusion for High-Quality 4D Generationcited by 44
  19. 2025WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actionscited by 43
  20. 2025Puppeteer: Rig and Animate Your 3D Modelscited by 41
  21. 2024DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priorscited by 40
  22. 2025Motion Anything: Any to Motion Generationcited by 40
  23. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 37
  24. 2025Free4D: Tuning-free 4D Scene Generation with Spatial-Temporal Consistencycited by 34
  25. 2025AnimateAnyMesh: A Feed-Forward 4D Foundation Model for Text-Driven Universal Mesh Animationcited by 31
  26. 20254DNeX: Feed-Forward 4D Generative Modeling Made Easycited by 31
  27. 2024Efficient Physics Simulation for 3D Scenes via MLLM-Guided Gaussian Splattingcited by 30
  28. 2024A Survey on Personalized Content Synthesis with Diffusion Modelscited by 29
  29. 2024A Comprehensive Survey on 3D Content Generationcited by 28
  30. 20244Real-Video: Learning Generalizable Photo-Realistic 4D Video Diffusioncited by 27
  31. 2025DreamArt: Generating Interactable Articulated Objects from a Single Imagecited by 25
  32. 2024Diffusion2: Dynamic 3D Content Generation via Score Composition of Video and Multi-view Diffusion Modelscited by 24
  33. 2024Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attentioncited by 24
  34. 2024Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perceptioncited by 23
  35. 2024Phy124: Fast Physics-Driven 4D Content Generation from a Single Imagecited by 22
  36. 2025FreeArt3D: Training-Free Articulated Object Generation using 3D Diffusioncited by 22
  37. 2023Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videoscited by 21
  38. 2025HiMoR: Monocular Deformable Gaussian Reconstruction with Hierarchical Motion Representationcited by 21

50 most cited of 113 All 113 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023A Unified Approach for Text- and Image-guided 4D Scene Generation
  2. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors
  3. 2024Diffusion Priors for Dynamic View Synthesis from Monocular Videos
  4. 2024Animate Your Motion: Turning Still Images into Dynamic Videos
  5. 2024CT4D: Consistent Text-to-4D Generation with Animatable Meshes
  6. 20244Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models
  7. 2025Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Models
  8. 2025Dynamic-eDiTor: Training-Free Text-Driven 4D Scene Editing with Multimodal Diffusion Transformer