mlsanityData to 7 Oct 2026

ViVid-1-to-3: Novel View Synthesis with Video Diffusion Models

arXiv 2312.01305 ↗First posted 3 Dec 2023cs.CV cs.AI cs.GRcited by 57

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Generating novel views of an object from a single image is a challenging task. It requires an understanding of the underlying 3D structure of the object from an image and rendering high-quality, spatially consistent new views. While recent methods for view synthesis based on diffusion have shown great progress, achieving consistency among various view estimates and at the same time abiding by the desired camera pose remains a critical problem yet to be solved. In this work, we demonstrate a strikingly simple method, where we utilize a pre-trained video diffusion model to solve this problem. Our key idea is that synthesizing a novel view could be reformulated as synthesizing a video of a camera going around the object of interest -- a scanning video -- which then allows us to leverage the powerful priors that a video diffusion model would have learned. Thus, to perform novel-view synthesis, we create a smooth camera trajectory to the target view that we wish to render, and denoise using both a view-conditioned diffusion model and a video diffusion model. By doing so, we obtain a highly consistent novel view synthesis, outperforming the state of the art.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 67 of its 79 references that are in the map. First posted on arXiv 3 Dec 2023.

Builds on

67 of 79 references.

Closest ancestors · 18

Same direction, newest first

  1. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  2. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
  3. 2023Consistent123: Improve Consistency for One Image to 3D Object Synthesis12 Oct
  4. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  5. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
  6. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
  7. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  8. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
  9. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
  10. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  11. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  12. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation14 Mar
6 more
  1. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models28 Dec
  2. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  3. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views29 Nov
  4. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  5. 2022Novel View Synthesis with Diffusion Models6 Oct
  6. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2017Neural Discrete Representation Learningcited by 4,650
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
37 more
  1. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  2. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  3. 2022Video Diffusion Modelscited by 2,280
  4. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  5. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  6. 2021Vision Transformers for Dense Predictioncited by 1,789
  7. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  8. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  9. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  10. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  11. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  12. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  13. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  14. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  15. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  16. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  17. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  18. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  19. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  20. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  21. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  22. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  23. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  24. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  25. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  26. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  27. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  28. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  29. 2020Free View Synthesiscited by 231
  30. 2015DeepStereo: Learning to Predict New Views from the World's Imagerycited by 185
  31. 2020GRF: Learning a General Radiance Field for 3D Representation and Renderingcited by 183
  32. 2020Stable View Synthesiscited by 165
  33. 2023Re-imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D, alleviate Janus problem and Beyondcited by 139
  34. 2023Unsupervised Semantic Correspondence Using Stable Diffusioncited by 120
  35. 2023ConsistNet: Enforcing 3D Consistency for Multi-view Images Diffusioncited by 65
  36. 2023SLiMe: Segment Like Mecited by 34
  37. 2022EpipolarNVS: leveraging on Epipolar geometry for single-image Novel View Synthesiscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 30; 2025: 17; 2026: 9.

57 papers.

Sort:

By area:

  1. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  2. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 143
  3. 2024IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generationcited by 84
  4. 2024SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglementcited by 65
  5. 2024Advances in 3D Generation: A Surveycited by 62
  6. 2024VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Modelscited by 56
  7. 2024StreetCrafter: Street View Synthesis with Controllable Video Diffusion Modelscited by 49
  8. 2025PhysAnimator: Physics-Guided Generative Cartoon Animationcited by 40
  9. 2025Matrix3D: Large Photogrammetry Model All-in-Onecited by 39
  10. 2024Diffusion Meets DAgger: Supercharging Eye-in-hand Imitation Learningcited by 34
  11. 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generationcited by 33
  12. 2024Ouroboros3D: Image-to-3D Generation via 3D-aware Recursive Diffusioncited by 31
38 more
  1. 2024Pathways on the Image Manifold: Image Editing via Video Generationcited by 27
  2. 2024SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Devicecited by 25
  3. 2024Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attentioncited by 24
  4. 2024Novel View Extrapolation with Video Diffusion Priorscited by 23
  5. 2026VideoGPA: Distilling Geometry Priors for 3D-Consistent Video Generationcited by 19
  6. 2024SplatFormer: Point Transformer for Robust 3D Gaussian Splattingcited by 15
  7. 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 14
  8. 2024A Survey on 3D Human Avatar Modeling -- From Reconstruction to Generationcited by 13
  9. 2025Edit360: 2D Image Edits to 3D Assets from Any Anglecited by 13
  10. 2024Envision3D: One Image to 3D with Anchor Views Interpolationcited by 12
  11. 2024Flex3D: Feed-Forward 3D Generation with Flexible Reconstruction Model and Input View Curationcited by 12
  12. 2025H3AE: High Compression, High Speed, and High Quality AutoEncoder for Video Diffusion Modelscited by 12
  13. 2024LiftImage3D: Lifting Any Single Image to 3D Gaussians with Video Generation Priorscited by 11
  14. 2023MoBluRF: Motion Deblurring Neural Radiance Fields for Blurry Monocular Videocited by 9
  15. 2024Zero-to-Hero: Enhancing Zero-Shot Novel View Synthesis via Attention Map Filteringcited by 9
  16. 2024A Lesson in Splats: Teacher-Guided Diffusion for 3D Gaussian Splats Generation with 2D Supervisioncited by 9
  17. 2025MVGBench: Comprehensive Benchmark for Multi-view Generation Modelscited by 9
  18. 2024FoundHand: Large-Scale Domain-Specific Learning for Controllable Hand Image Generationcited by 8
  19. 2025Voost: A Unified and Scalable Diffusion Transformer for Bidirectional Virtual Try-On and Try-Offcited by 8
  20. 2024KFC-W: Generating 3D-Consistent Videos from Unposed Internet Photoscited by 7
  21. 2024GECO: Generative Image-to-3D within a SECOndcited by 6
  22. 2025DIMO: Diverse 3D Motion Generation for Arbitrary Objectscited by 6
  23. 2024GeoGS3D: Single-view 3D Reconstruction via Geometric-aware Diffusion Model and Gaussian Splattingcited by 5
  24. 2024Bootstrap3D: Improving Multi-view Diffusion Model with Synthetic Datacited by 5
  25. 2024Free-viewpoint Human Animation with Pose-correlated Reference Selectioncited by 5
  26. 2025GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstructioncited by 4
  27. 2025WAVE: Warp-Based View Guidance for Consistent Novel View Synthesis Using a Single Imagecited by 3
  28. 2024Tex4D: Zero-shot 4D Scene Texturing with Video Diffusion Modelscited by 2
  29. 2025Track, Inpaint, Resplat: Subject-driven 3D and 4D Generation with Progressive Texture Infillingcited by 2
  30. 2025DreamPose3D: Hallucinative Diffusion with Prompt Learning for 3D Human Pose Estimationcited by 2
  31. 2024View Transformation Robustness for Multi-View 3D Object Reconstruction with Reconstruction Error-Guided View Selectioncited by 1
  32. 2025ViewMask-1-to-3: Multi-View Consistent Image Generation via Multimodal Discrete Diffusion Modelscited by 1
  33. 2026ERGO: Excess-Risk-Guided Optimization for High-Fidelity Monocular 3D Gaussian Splattingcited by 1
  34. 2026OrbitNVS: Harnessing Video Diffusion Priors for Novel View Synthesiscited by 1
  35. 2026CityRAG: Stepping Into a City via Spatially-Grounded Video Generationcited by 1
  36. 2024Focus on Neighbors and Know the Whole: Towards Consistent Dense Multiview Text-to-Image Generator for 3D Creationcited by 0
  37. 2025Text To 3D Object Generation For Scalable Room Assemblycited by 0
  38. 2025EgoAnimate: Generating Human Animations from Egocentric top-down Viewscited by 0

50 most cited of 57 All 57 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizer
  2. 2023Consistent View Synthesis with Pose-Guided Diffusion Models
  3. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis
  4. 2025DT-NVS: Diffusion Transformers for Novel View Synthesis
  5. 2024Novel View Synthesis with Pixel-Space Diffusion Models
  6. 2023Efficient-3DiM: Learning a Generalizable Single-image Novel-view Synthesizer in One Day
  7. 2025Geometric Consistency Refinement for Single Image Novel View Synthesis via Test-Time Adaptation of Diffusion Models
  8. 2024FSViewFusion: Few-Shots View Generation of Novel Objects