mlsanityData to 14 Oct 2026

MVDD: Multi-View Depth Diffusion Models

arXiv 2312.04875 ↗First posted 8 Dec 2023cs.CVcited by 10

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Denoising diffusion models have demonstrated outstanding results in 2D image generation, yet it remains a challenge to replicate its success in 3D shape generation. In this paper, we propose leveraging multi-view depth, which represents complex 3D shapes in a 2D data format that is easy to denoise. We pair this representation with a diffusion model, MVDD, that is capable of generating high-quality dense point clouds with 20K+ points with fine-grained details. To enforce 3D consistency in multi-view depth, we introduce an epipolar line segment attention that conditions the denoising step for a view on its neighboring views. Additionally, a depth fusion module is incorporated into diffusion steps to further ensure the alignment of depth maps. When augmented with surface reconstruction, MVDD can also produce high-quality 3D meshes. Furthermore, MVDD stands out in other tasks such as depth completion, and can serve as a 3D prior, significantly boosting many downstream tasks, such as GAN inversion. State-of-the-art results from extensive experiments demonstrate MVDD's excellent ability in 3D shape generation, depth completion, and its potential as a 3D prior for downstream tasks.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 47 of its 58 references that are in the map. First posted on arXiv 8 Dec 2023.

Builds on

47 of 58 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3D4 Oct
  2. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  3. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  4. 2023TextMesh: Generation of Realistic 3D Meshes From Text Prompts24 Apr
  5. 2023Consistent View Synthesis with Pose-Guided Diffusion Models30 Mar
  6. 20223D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Models1 Dec
  7. 2022Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functions24 Nov
  8. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  9. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  10. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  11. 2022Generative Adversarial Networkscited by 5,627
  12. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
25 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  2. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  3. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,323
  4. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  5. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  6. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 788
  7. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  8. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  9. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 488
  10. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  11. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  12. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 333
  13. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 306
  14. 2016Revisiting Classifier Two-Sample Testscited by 268
  15. 2020Learning Gradient Fields for Shape Generationcited by 245
  16. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  17. 2023MeshDiffusion: Score-based Generative 3D Mesh Modelingcited by 199
  18. 2021Shape As Points: A Differentiable Poisson Solvercited by 187
  19. 20193D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutionscited by 173
  20. 2020SoftFlow: Probabilistic Framework for Normalizing Flow on Manifoldscited by 98
  21. 2020Discrete Point Flow Networks for Efficient Point Cloud Generationcited by 87
  22. 2018An empirical study on evaluation metrics of generative adversarial networkscited by 81
  23. 2021SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Datacited by 78
  24. 2018Learning to Reconstruct Shapes from Unseen Classescited by 60
  25. 2022Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversioncited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 1.

10 papers.

  1. 2024Direct and Explicit 3D Generation from a Single Imagecited by 6
  2. 2025DiffPCN: Latent Diffusion Model Based on Multi-view Depth Images for Point Cloud Completioncited by 5
  3. 2024Jointly Generating Multi-view Consistent PBR Textures using Collaborative Controlcited by 4
  4. 2025GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstructioncited by 4
  5. 2025Stable-Sim2Real: Exploring Simulation of Real-Captured 3D Data with Two-Stage Depth Diffusioncited by 2
  6. 2025IM-Portrait: Learning 3D-aware Video Diffusion for Photorealistic Talking Heads from Monocular Videoscited by 1
  7. 2025BridgeShape: Latent Diffusion Schr"odinger Bridge for 3D Shape Completioncited by 1
  8. 2025MVRoom: Controllable 3D Indoor Scene Generation with Multi-View Diffusion Modelscited by 1
  9. 2024GPN: Generative Point-based NeRFcited by 0
  10. 2026STEREOFLOW: Progressive Stereo Matching with StereoDiT and Transition Flow Matchingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MVD2: Efficient Multiview 3D Reconstruction for Multiview Diffusion
  2. 2025Lightweight and Accurate Multi-View Stereo with Confidence-Aware Diffusion Model
  3. 2024MVDiff: Scalable and Flexible Multi-View Diffusion for 3D Object Reconstruction from Single-View
  4. 2024MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generation
  5. 2023DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimation
  6. 2024MVGenMaster: Scaling Multi-View Generation from Any Image via 3D Priors Enhanced Diffusion Model
  7. 2023DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generation
  8. 2024FiffDepth: Feed-forward Transformation of Diffusion-Based Generators for Detailed Depth Estimation