mlsanityData to 7 Oct 2026

SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency

arXiv 2407.17470 ↗First posted 24 Jul 2024cs.CVcited by 143

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present Stable Video 4D (SV4D), a latent video diffusion model for multi-frame and multi-view consistent dynamic 3D content generation. Unlike previous methods that rely on separately trained generative models for video generation and novel view synthesis, we design a unified diffusion model to generate novel view videos of dynamic 3D objects. Specifically, given a monocular reference video, SV4D generates novel views for each video frame that are temporally consistent. We then use the generated novel view videos to optimize an implicit 4D representation (dynamic NeRF) efficiently, without the need for cumbersome SDS-based optimization used in most prior works. To train our unified novel view video generation model, we curate a dynamic 3D object dataset from the existing Objaverse dataset. Extensive experimental results on multiple datasets and user studies demonstrate SV4D's state-of-the-art performance on novel-view video synthesis as well as 4D generation compared to prior works.

Key facts. Cited by 143 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 82 of its 91 references that are in the map. First posted on arXiv 24 Jul 2024.

Builds on

82 of 91 references.

Closest ancestors · 33

Same direction, newest first

  1. 2024Animate3D: Animating Any 3D Model with Multi-view Video Diffusion16 Jul
  2. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generation31 May
  3. 2024EG4D: Explicit Generation of 4D Object without Score Distillation28 May
  4. 2024Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Models26 May
  5. 2024Diffusion2: Dynamic 3D Content Generation via Score Composition of Video and Multi-view Diffusion Models2 Apr
  6. 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians22 Mar
  7. 2024VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models18 Mar
  8. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion18 Mar
  9. 2024V3D: Video Diffusion Models are Effective 3D Generators11 Mar
  10. 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistency28 Dec
  11. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models21 Dec
  12. 2023ViVid-1-to-3: Novel View Synthesis with Video Diffusion Models3 Dec
21 more
  1. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation2 Dec
  2. 2023StableDreamer: Taming Noisy Score Distillation Sampling for Text-to-3D2 Dec
  3. 2023DreamPropeller: Supercharge Text-to-3D Generation with Parallel Sampling28 Nov
  4. 2023Animate124: Animating One Image to 4D Dynamic Scene24 Nov
  5. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion14 Nov
  6. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model10 Nov
  7. 2023ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Image27 Oct
  8. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior25 Oct
  9. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  10. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
  11. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models12 Oct
  12. 2023Consistent123: Improve Consistency for One Image to 3D Object Synthesis12 Oct
  13. 2023Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models4 Oct
  14. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3D4 Oct
  15. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  16. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  17. 2023HoloFusion: Towards Photo-realistic 3D Generative Modeling28 Aug
  18. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
  19. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  20. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  21. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 49

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  6. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  8. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  9. 2022Video Diffusion Modelscited by 2,280
  10. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  11. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  12. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
37 more
  1. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  2. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  3. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  4. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  5. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  6. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  7. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  8. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  9. 2023Adversarial Diffusion Distillationcited by 741
  10. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  11. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 582
  12. 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 467
  13. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 385
  14. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  15. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  16. 2023Text-to-3D using Gaussian Splattingcited by 299
  17. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  18. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 257
  19. 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 227
  20. 2023Text-To-4D Dynamic Scene Generationcited by 204
  21. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 191
  22. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 184
  23. 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
  24. 2023PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Predictioncited by 165
  25. 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 143
  26. 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 138
  27. 2023Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Videocited by 124
  28. 2024GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creationcited by 99
  29. 2019The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentationcited by 95
  30. 2022SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collectionscited by 89
  31. 2024IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generationcited by 84
  32. 2023LEAP: Liberate Sparse-view 3D Modeling from Camera Posescited by 65
  33. 2024Efficient4D: Fast Dynamic 3D Object Generation from a Single-view Videocited by 44
  34. 2024Vivid-ZOO: Multi-View Video Generation with Diffusion Modelcited by 40
  35. 2024Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfelscited by 38
  36. 2023TOSS:High-quality Text-guided Novel View Synthesis from a Single Imagecited by 26
  37. 2023Enhancing High-Resolution 3D Generation through Pixel-wise Gradient Clippingcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 23; 2025: 69; 2026: 51.

143 papers.

Sort:

By area:

  1. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
  2. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  3. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 139
  4. 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 98
  5. 2024SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpointscited by 92
  6. 2025MEt3R: Measuring Multi-View Consistency in Generated Imagescited by 91
  7. 2025TesserAct: Learning 4D Embodied World Modelscited by 90
  8. 2024SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglementcited by 65
  9. 2024GenXD: Generating Any 3D and 4D Scenescited by 49
  10. 20254DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videoscited by 46
  11. 2025SV4D 2.0: Enhancing Spatio-Temporal Consistency in Multi-View Video Diffusion for High-Quality 4D Generationcited by 44
  12. 2025CamCloneMaster: Enabling Reference-based Camera Control for Video Generationcited by 44
38 more
  1. 2025World-in-World: World Models in a Closed-Loop Worldcited by 44
  2. 2025Puppeteer: Rig and Animate Your 3D Modelscited by 41
  3. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 37
  4. 2024DreamDrive: Generative 4D Scene Modeling from Street View Imagescited by 35
  5. 2025Reangle-A-Video: 4D Video Generation as Video-to-Video Translationcited by 35
  6. 2025Free4D: Tuning-free 4D Scene Generation with Spatial-Temporal Consistencycited by 34
  7. 2025Gaussian Variation Field Diffusion for High-fidelity Video-to-4D Synthesiscited by 33
  8. 2024Puppet-Master: Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamicscited by 28
  9. 20244Real-Video: Learning Generalizable Photo-Realistic 4D Video Diffusioncited by 27
  10. 2024Generative Photography: Scene-Consistent Camera Control for Realistic Text-to-Image Synthesiscited by 26
  11. 2025DreamArt: Generating Interactable Articulated Objects from a Single Imagecited by 25
  12. 2024Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attentioncited by 24
  13. 2024Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perceptioncited by 23
  14. 2025Geometry-aware 4D Video Generation for Robot Manipulationcited by 23
  15. 2025ShapeGen4D: Towards High Quality 4D Shape Generation from Videoscited by 20
  16. 2025GAS: Generative Avatar Synthesis from a Single Imagecited by 19
  17. 2025Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos with Spatio-Temporal Diffusion Modelscited by 17
  18. 2024SplatFormer: Point Transformer for Robust 3D Gaussian Splattingcited by 15
  19. 2024Using Diffusion Priors for Video Amodal Segmentationcited by 15
  20. 20244D Gaussian Splatting: Modeling Dynamic Scenes with Native 4D Primitivescited by 15
  21. 2025Zero4D: Training-Free 4D Video Generation From Single Video Using Off-the-Shelf Video Diffusioncited by 15
  22. 2025AnimaX: Animating the Inanimate in 3D with Joint Video-Pose Diffusion Modelscited by 15
  23. 2026Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesiscited by 15
  24. 2026ActionMesh: Animated 3D Mesh Generation with Temporal 3D Diffusioncited by 15
  25. 2026Vista4D: Video Reshooting with 4D Point Cloudscited by 15
  26. 2025Articulated Kinematics Distillation from Video Diffusion Modelscited by 13
  27. 20254D-LRM: Large Space-Time Reconstruction Model From and To Any View at Any Timecited by 13
  28. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  29. 2025Dynamic View Synthesis as an Inverse Problemcited by 12
  30. 20254Real-Video-V2: Fused View-Time Attention and Feedforward Reconstruction for 4D Scene Generationcited by 12
  31. 2026Mesh4D: 4D Mesh Reconstruction and Tracking from Monocular Videocited by 12
  32. 2025Drive Any Mesh: 4D Latent Diffusion for Mesh Deformation from Videocited by 11
  33. 2025Not All Frame Features Are Equal: Video-to-4D Generation via Decoupling Dynamic-Static Featurescited by 10
  34. 2025TACO: Taming Diffusion for in-the-wild Video Amodal Completioncited by 10
  35. 2025Can Video Diffusion Model Reconstruct 4D Geometry?cited by 10
  36. 2025DiVE: Efficient Multi-View Driving Scenes Generation Based on Video Diffusion Transformercited by 10
  37. 2026Action Images: End-to-End Policy Learning via Multiview Video Generationcited by 10
  38. 2024Motion-2-To-3: Leveraging 2D Motion Data for 3D Motion Generationscited by 9

50 most cited of 143 All 143 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Vid3D: Synthesis of Dynamic 3D Scenes using 2D Video Diffusion
  2. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models
  3. 20254DNeX: Feed-Forward 4D Generative Modeling Made Easy
  4. 20244Dynamic: Text-to-4D Generation with Hybrid Priors
  5. 2026Beyond Pixels: From Video Priors to 4D Worlds
  6. 20244Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models
  7. 2026Full-4D: Generating Full-Scope 4D Scenes from a Single-View Video
  8. 2025MVTokenFlow: High-quality 4D Content Generation using Multiview Token Flow