mlsanityData to 14 Oct 2026

MVDiff: Scalable and Flexible Multi-View Diffusion for 3D Object Reconstruction from Single-View

arXiv 2405.03894 ↗First posted 6 May 2024cs.CV cs.LGcited by 6

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Generating consistent multiple views for 3D reconstruction tasks is still a challenge to existing image-to-3D diffusion models. Generally, incorporating 3D representations into diffusion model decrease the model's speed as well as generalizability and quality. This paper proposes a general framework to generate consistent multi-view images from single image or leveraging scene representation transformer and view-conditioned diffusion model. In the model, we introduce epipolar geometry constraints and multi-view attention to enforce 3D consistency. From as few as one image input, our model is able to generate 3D meshes surpassing baselines methods in evaluation metrics, including PSNR, SSIM and LPIPS.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 40 of its 46 references that are in the map. First posted on arXiv 6 May 2024.

Builds on

40 of 46 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024EscherNet: A Generative Model for Scalable View Synthesis6 Feb
  2. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  3. 2023Large-Vocabulary 3D Diffusion Model with Transformer14 Sep
  4. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  5. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  6. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
  7. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  8. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  9. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion12 Dec
  10. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusion2 Dec
  11. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 29

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2022Classifier-Free Diffusion Guidancecited by 5,847
  8. 2022Generative Adversarial Networkscited by 5,627
  9. 2017Neural Discrete Representation Learningcited by 4,712
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  11. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  12. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
17 more
  1. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  2. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  4. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  5. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  6. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  7. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  8. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 788
  9. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
  10. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  11. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 612
  12. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  13. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 334
  14. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 227
  15. 2023PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Predictioncited by 165
  16. 2023LEAP: Liberate Sparse-view 3D Modeling from Camera Posescited by 65
  17. 2022FvOR: Robust Joint Shape and Pose Optimization for Few-view Object Reconstructioncited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025Eval3D: Interpretable and Fine-grained Evaluation for 3D Generationcited by 16
  2. 2025RayPose: Ray Bundling Diffusion for Template Views in Unseen 6D Object Pose Estimationcited by 4
  3. 2026SceneLinker: Compositional 3D Scene Generation via Semantic Scene Graph from RGB Sequencescited by 3
  4. 2026StructSplat: Generalizable 3D Gaussian Splatting from Uncalibrated Sparse Viewscited by 2
  5. 2025SatDreamer360: Multiview-Consistent Generation of Ground-Level Scenes from Satellite Imagerycited by 0
  6. 2025Novel View Synthesis using DDIM Inversioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MVD2: Efficient Multiview 3D Reconstruction for Multiview Diffusion
  2. 2024MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstruction
  3. 2023EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusion
  4. 2024DiffTF++: 3D-aware Diffusion Transformer for Large-Vocabulary 3D Generation
  5. 2023Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models
  6. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion
  7. 2024VistaDream: Sampling multiview consistent images for single-view scene reconstruction
  8. 2024Hi3D: Pursuing High-Resolution Image-to-3D Generation with Video Diffusion Models