mlsanityData to 7 Oct 2026

V3D: Video Diffusion Models are Effective 3D Generators

arXiv 2403.06738 ↗First posted 11 Mar 2024cs.CVcited by 88

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Automatic 3D generation has recently attracted widespread attention. Recent methods have greatly accelerated the generation speed, but usually produce less-detailed objects due to limited model capacity or 3D data. Motivated by recent advancements in video diffusion models, we introduce V3D, which leverages the world simulation capacity of pre-trained video diffusion models to facilitate 3D generation. To fully unleash the potential of video diffusion to perceive the 3D world, we further introduce geometrical consistency prior and extend the video diffusion model to a multi-view consistent 3D generator. Benefiting from this, the state-of-the-art video diffusion model could be fine-tuned to generate 360degree orbit frames surrounding an object given a single image. With our tailored reconstruction pipelines, we can generate high-quality meshes or 3D Gaussians within 3 minutes. Furthermore, our method can be extended to scene-level novel view synthesis, achieving precise control over the camera path with sparse input views. Extensive experiments demonstrate the superior performance of the proposed approach, especially in terms of generation quality and multi-view consistency. Our code is available at https://github.com/heheyas/V3D

Key facts. Cited by 88 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 104 of its 115 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

104 of 115 references.

Closest ancestors · 30

Same direction, newest first

  1. 2024MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstruction20 Feb
  2. 2023Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models21 Dec
  3. 2023Mosaic-SDF for 3D Generative Models14 Dec
  4. 2023UniDream: Unifying Diffusion Priors for Relightable Text-to-3D Generation14 Dec
  5. 2023ReconFusion: 3D Reconstruction with Diffusion Priors5 Dec
  6. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation2 Dec
  7. 2023Direct2.5: Diverse Text-to-3D Generation via Multi-view 2.5D Diffusion27 Nov
  8. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion14 Nov
  9. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model10 Nov
  10. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior25 Oct
  11. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  12. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
18 more
  1. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3D4 Oct
  2. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  3. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  4. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
  5. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
  6. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Data13 Jun
  7. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  8. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
  9. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Images29 Mar
  10. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
  11. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  12. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  13. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation14 Mar
  14. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  15. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion12 Dec
  16. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  17. 2022Novel View Synthesis with Diffusion Models6 Oct
  18. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 74

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  4. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  6. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  7. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  8. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  9. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  10. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  11. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  12. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
38 more
  1. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  2. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  3. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  4. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  5. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  6. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  7. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  8. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  9. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  10. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  11. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  12. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
  13. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  14. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
  15. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  16. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  17. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  18. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  19. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  20. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  21. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  22. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  23. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  24. 2023GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splattingcited by 331
  25. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 308
  26. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  27. 2023Text-to-3D using Gaussian Splattingcited by 299
  28. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  29. 2023FSGS: Real-Time Few-shot View Synthesis using Gaussian Splattingcited by 278
  30. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  31. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 257
  32. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  33. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  34. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 254
  35. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 222
  36. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
  37. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  38. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 213

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 41; 2025: 36; 2026: 11.

88 papers.

Sort:

By area:

  1. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  2. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 143
  3. 2024MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformerscited by 140
  4. 2024ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Modelcited by 139
  5. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 139
  6. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
  7. 2024Training-free Camera Control for Video Generationcited by 119
  8. 20243DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priorscited by 106
  9. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
  10. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 99
  11. 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 98
  12. 2024Scaling Mesh Generation via Compressive Tokenizationcited by 71
38 more
  1. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  2. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learningcited by 62
  3. 2024PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Modelscited by 60
  4. 2024PivotMesh: Generic 3D Mesh Generation via Pivot Vertices Guidancecited by 47
  5. 2025PhysAnimator: Physics-Guided Generative Cartoon Animationcited by 40
  6. 2024Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfelscited by 38
  7. 2025GS-DiT: Advancing Video Generation with Pseudo 4D Gaussian Fields through Efficient Dense 3D Point Trackingcited by 37
  8. 2025ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understandingcited by 34
  9. 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generationcited by 33
  10. 2025Dynamic Camera Poses and Where to Find Themcited by 26
  11. 2024Diffusion2: Dynamic 3D Content Generation via Score Composition of Video and Multi-view Diffusion Modelscited by 24
  12. 2024Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attentioncited by 24
  13. 2025Epipolar Geometry Improves Video Generation Modelscited by 22
  14. 2024GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Detailscited by 19
  15. 2024Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabricationcited by 19
  16. 2024SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrixcited by 19
  17. 2024Phidias: A Generative Model for Creating 3D Content from Text, Image, and 3D Conditions with Reference-Augmented Diffusioncited by 19
  18. 2025Dress-1-to-3: Single Image to Simulation-Ready 3D Outfit with Diffusion Prior and Differentiable Physicscited by 19
  19. 2025WorldGen: From Text to Traversable and Interactive 3D Worldscited by 18
  20. 2024Atlas Gaussians Diffusion for 3D Generationcited by 15
  21. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  22. 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 14
  23. 2024A Survey On Text-to-3D Contents Generation In The Wildcited by 13
  24. 2024A Survey on 3D Human Avatar Modeling -- From Reconstruction to Generationcited by 13
  25. 2025Scene Splatter: Momentum 3D Scene Generation from Single Image with Video Diffusion Modelcited by 13
  26. 2025Edit360: 2D Image Edits to 3D Assets from Any Anglecited by 13
  27. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  28. 2024Large Point-to-Gaussian Model for Image-to-3D Generationcited by 10
  29. 2024Edify 3D: Scalable High-Quality 3D Asset Generationcited by 10
  30. 20254DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Modelscited by 10
  31. 2024Semantic Score Distillation Sampling for Compositional Text-to-3D Generationcited by 9
  32. 2024MVReward: Better Aligning and Evaluating Multi-View Diffusion Models with Human Preferencescited by 9
  33. 2025MVGBench: Comprehensive Benchmark for Multi-view Generation Modelscited by 9
  34. 2025CoRe3D: Collaborative Reasoning as a Foundation for 3D Intelligencecited by 9
  35. 2025How Much 3D Do Video Foundation Models Encode?cited by 9
  36. 20253D Stylization via Large Reconstruction Modelcited by 8
  37. 2025MeshRipple: Structured Autoregressive Generation of Artist-Meshescited by 8
  38. 2024KFC-W: Generating 3D-Consistent Videos from Unposed Internet Photoscited by 7

50 most cited of 88 All 88 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion
  2. 2024Hi3D: Pursuing High-Resolution Image-to-3D Generation with Video Diffusion Models
  3. 2025VideoFrom3D: 3D Scene Video Generation via Complementary Image and Video Diffusion Models
  4. 2025View-Consistent Diffusion Representations for 3D-Consistent Video Generation
  5. 20253D Scene Generation: A Survey
  6. 2025VideoScene: Distilling Video Diffusion Model to Generate 3D Scenes in One Step
  7. 20243D-Adapter: Geometry-Consistent Multi-View Diffusion for High-Quality 3D Generation
  8. 2026Efficient Camera-Controlled Video Generation of Static Scenes via Sparse Diffusion and 3D Rendering