mlsanityData to 7 Oct 2026

SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion

arXiv 2403.12008 ↗First posted 18 Mar 2024cs.CVcited by 364

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present Stable Video 3D (SV3D) -- a latent video diffusion model for high-resolution, image-to-multi-view generation of orbital videos around a 3D object. Recent work on 3D generation propose techniques to adapt 2D generative models for novel view synthesis (NVS) and 3D optimization. However, these methods have several disadvantages due to either limited views or inconsistent NVS, thereby affecting the performance of 3D object generation. In this work, we propose SV3D that adapts image-to-video diffusion model for novel multi-view synthesis and 3D generation, thereby leveraging the generalization and multi-view consistency of the video models, while further adding explicit camera control for NVS. We also propose improved 3D optimization techniques to use SV3D and its NVS outputs for image-to-3D generation. Extensive experimental results on multiple datasets with 2D and 3D metrics as well as user study demonstrate SV3D's state-of-the-art performance on NVS as well as 3D reconstruction compared to prior works.

Key facts. Cited by 364 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 53 of its 61 references that are in the map. First posted on arXiv 18 Mar 2024.

Builds on

53 of 61 references.

Closest ancestors · 17

Same direction, newest first

  1. 2024EscherNet: A Generative Model for Scalable View Synthesis6 Feb
  2. 2023ViVid-1-to-3: Novel View Synthesis with Video Diffusion Models3 Dec
  3. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion14 Nov
  4. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  5. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
  6. 2023Consistent123: Improve Consistency for One Image to 3D Object Synthesis12 Oct
  7. 2023Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models4 Oct
  8. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  9. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  10. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
  11. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
  12. 2023ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collections7 Jun
5 more
  1. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
  2. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  3. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  4. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  5. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  11. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  12. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
24 more
  1. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  2. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  4. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  5. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  6. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  7. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  8. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  9. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  10. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  11. 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 467
  12. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  13. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 446
  14. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  15. 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 332
  16. 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 324
  17. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  18. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  19. 2022Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoisingcited by 163
  20. 2022SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collectionscited by 89
  21. 2024IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generationcited by 84
  22. 2023Free3D: Consistent Novel View Synthesis without 3D Representationcited by 72
  23. 2023ConsistNet: Enforcing 3D Consistency for Multi-view Images Diffusioncited by 65
  24. 2024HexaGen3D: StableDiffusion is just one step away from Fast and Diverse Text-to-3D Generationcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 113; 2025: 166; 2026: 85.

364 papers.

Sort:

By area:

  1. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  2. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  3. 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 275
  4. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
  5. 2024LVSM: A Large View Synthesis Model with Minimal 3D Inductive Biascited by 178
  6. 2024CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generationcited by 163
  7. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  8. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  9. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 143
  10. 2024ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Modelcited by 139
  11. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 139
  12. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
38 more
  1. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representationcited by 123
  2. 2024Training-free Camera Control for Video Generationcited by 119
  3. 2024Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesiscited by 116
  4. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 113
  5. 2024MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Viewscited by 111
  6. 2024MV-Adapter: Multi-view Consistent Image Generation Made Easycited by 111
  7. 20243DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priorscited by 106
  8. 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 101
  9. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 99
  10. 2024Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Modelscited by 99
  11. 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 98
  12. 2025MEt3R: Measuring Multi-View Consistency in Generated Imagescited by 91
  13. 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assetscited by 85
  14. 2024NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizercited by 79
  15. 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 77
  16. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
  17. 2024Scaling Mesh Generation via Compressive Tokenizationcited by 71
  18. 2024Not All Noises Are Created Equally:Diffusion Noise Selection and Optimizationcited by 70
  19. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 66
  20. 2024SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglementcited by 65
  21. 20244Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Modelscited by 64
  22. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learningcited by 62
  23. 2025Bolt3D: Generating 3D Scenes in Secondscited by 57
  24. 2024SCube: Instant Large-Scale Scene Reconstruction using VoxSplatscited by 56
  25. 2024Animate3D: Animating Any 3D Model with Multi-view Video Diffusioncited by 55
  26. 2024MeshFormer: High-Quality Mesh Generation with 3D-Guided Reconstruction Modelcited by 55
  27. 2024IDOL: Instant Photorealistic 3D Human Creation from a Single Imagecited by 55
  28. 2024HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priorscited by 54
  29. 2025Amodal3R: Amodal 3D Reconstruction from Occluded 2D Imagescited by 52
  30. 2024GenXD: Generating Any 3D and 4D Scenescited by 49
  31. 2024StreetCrafter: Street View Synthesis with Controllable Video Diffusion Modelscited by 49
  32. 2025DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generationcited by 49
  33. 2024PivotMesh: Generic 3D Mesh Generation via Pivot Vertices Guidancecited by 47
  34. 2024EG4D: Explicit Generation of 4D Object without Score Distillationcited by 45
  35. 2024Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Modelscited by 44
  36. 2025SV4D 2.0: Enhancing Spatio-Temporal Consistency in Multi-View Video Diffusion for High-Quality 4D Generationcited by 44
  37. 2025World-in-World: World Models in a Closed-Loop Worldcited by 44
  38. 2025PhysAnimator: Physics-Guided Generative Cartoon Animationcited by 40

50 most cited of 364 All 364 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024V3D: Video Diffusion Models are Effective 3D Generators
  2. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis
  3. 2025VideoFrom3D: 3D Scene Video Generation via Complementary Image and Video Diffusion Models
  4. 2024VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models
  5. 20243D-free meets 3D priors: Novel View Synthesis from a Single Image with Pretrained Diffusion Guidance
  6. 2026Gen3R: 3D Scene Generation Meets Feed-Forward Reconstruction
  7. 2024MVDiff: Scalable and Flexible Multi-View Diffusion for 3D Object Reconstruction from Single-View
  8. 2023DreamComposer: Controllable 3D Object Generation via Multi-View Conditions