mlsanityData to 14 Oct 2026

MVGenMaster: Scaling Multi-View Generation from Any Image via 3D Priors Enhanced Diffusion Model

arXiv 2411.16157 ↗First posted 25 Nov 2024cs.CVcited by 32

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We introduce MVGenMaster, a multi-view diffusion model enhanced with 3D priors to address versatile Novel View Synthesis (NVS) tasks. MVGenMaster leverages 3D priors that are warped using metric depth and camera poses, significantly enhancing both generalization and 3D consistency in NVS. Our model features a simple yet effective pipeline that can generate up to 100 novel views conditioned on variable reference views and camera poses with a single forward process. Additionally, we have developed a comprehensive large-scale multi-view image dataset called MvD-1M, comprising up to 1.6 million scenes, equipped with well-aligned metric depth to train MVGenMaster. Moreover, we present several training and model modifications to strengthen the model with scaled-up datasets. Extensive evaluations across in- and out-of-domain benchmarks demonstrate the effectiveness of our proposed method and data formulation. Models and codes will be released at https://github.com/ewrfcas/MVGenMaster/.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 80 of its 95 references that are in the map. First posted on arXiv 25 Nov 2024.

Builds on

80 of 95 references.

Closest ancestors · 22

Same direction, newest first

  1. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis3 Sep
  2. 2024ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Model29 Aug
  3. 2024Director3D: Real-world Camera Trajectory and 3D Scene Generation from Text25 Jun
  4. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Models16 May
  5. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusion10 Apr
  6. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion18 Mar
  7. 2024Denoising Diffusion via Image-Based Rendering5 Feb
  8. 2023ReconFusion: 3D Reconstruction with Diffusion Priors5 Dec
  9. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation2 Dec
  10. 2023ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Image27 Oct
  11. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  12. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
10 more
  1. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3D4 Oct
  2. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  3. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  4. 2023HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance30 May
  5. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  6. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
  7. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  8. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  9. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  10. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 58

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  8. 2022Scalable Diffusion Models with Transformerscited by 6,054
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  11. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  12. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
38 more
  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  2. 2022Video Diffusion Modelscited by 2,305
  3. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  4. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  5. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  6. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  7. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  8. 2024Depth Anything V2cited by 1,422
  9. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  10. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
  11. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
  12. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
  13. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  14. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 684
  15. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 671
  16. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
  17. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  18. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  19. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 566
  20. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
  21. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 484
  22. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 471
  23. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 404
  24. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 365
  25. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 311
  26. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 310
  27. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  28. 2020Query-Key Normalization for Transformerscited by 280
  29. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 258
  30. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 255
  31. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  32. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 243
  33. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 220
  34. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 214
  35. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 202
  36. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
  37. 2023WonderJourney: Going from Anywhere to Everywherecited by 158
  38. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 116

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 21.

32 papers.

Sort:

By area:

  1. 2025Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generationcited by 77
  2. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 27
  3. 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modelingcited by 21
  4. 2025Positional Encoding Fieldcited by 20
  5. 2026WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric Memoriescited by 18
  6. 2025ORV: 4D Occupancy-centric Robot Video Generationcited by 17
  7. 2025SpatialCrafter: Unleashing the Imagination of Video Diffusion Models for Scene Reconstruction from Limited Observationscited by 16
  8. 2026Repurposing Geometric Foundation Models for Multi-view Diffusioncited by 15
  9. 2025CAMEO: Correspondence-Attention Alignment for Multi-View Diffusion Modelscited by 10
  10. 2025PostCam: Camera-Controllable Novel-View Video Generation with Query-Shared Cross-Attentioncited by 6
  11. 2025View-Consistent Diffusion Representations for 3D-Consistent Video Generationcited by 4
  12. 2025GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstructioncited by 4
20 more
  1. 2026GeoNVS: Geometry Grounded Video Diffusion for Novel View Synthesiscited by 3
  2. 2025CloseUpShot: Close-up Novel View Synthesis from Sparse-views via Point-conditioned Diffusion Modelcited by 2
  3. 2026GeoFlow: Enforcing Implicit Geometric Consistency in Video Generationcited by 2
  4. 2026HarmoView: Harmonizing Multi-View Constraints for Identity-Consistent Video Generationcited by 1
  5. 2026PanoImager: Geometry-Guided Novel View Synthesis and Reconstruction from Sparse Panoramic Viewscited by 1
  6. 2025ARSS: Taming Decoder-only Autoregressive Visual Generation for View Synthesis From Single Viewcited by 0
  7. 2026Towards Realistic and Consistent Orbital Video Generation via 3D Foundation Priorscited by 0
  8. 2026Can These Views Be One Scene? Evaluating Multiview 3D Consistency when 3D Foundation Models Hallucinatecited by 0
  9. 2026DeblurNVS: Geometric Latent Diffusion for Novel View Synthesis from Sparse Motion-Blurred Imagescited by 0
  10. 2026GeoFace: Consistent Multi-View Face Generation with Geometry-Constrained Diffusioncited by 0
  11. 2026WarpHammer: Densifying Scene Warps with 3D Object Priors for Extreme View Synthesiscited by 0
  12. 2026SplatGuide: Geometric Priors from 3D Gaussians for Pose-Free Novel View Synthesiscited by 0
  13. 2026Puffin-World: Scaling a Unified Multimodal Model with Native 3D World Statescited by 0
  14. 2026Reflection-aware Generative Novel View Synthesiscited by 0
  15. 2026ORCA: Occlusion-Aware Refinement and Completion for Novel View Synthesiscited by 0
  16. 2026M3GD: Multi-Modal Multi-View Geometric Diffusion for Camera--LiDAR Novel View Synthesiscited by 0
  17. 2026Spackle: Completing Large View Single Image NVS with Adaptive Gaussianscited by 0
  18. 2026GeoVerse: World-Consistent Novel View Synthesis in Geometric Latent Spacecited by 0
  19. 2026PartiCam: Camera Controlled Video Generation with Reward Guidancecited by 0
  20. 2026Can Learning to Answer Spatial Questions Improve Novel-View Generation?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MVD2: Efficient Multiview 3D Reconstruction for Multiview Diffusion
  2. 2024MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generation
  3. 2024MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstruction
  4. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Model
  5. 2025MV-Performer: Taming Video Diffusion Model for Faithful and Synchronized Multi-view Performer Synthesis
  6. 2022Novel View Synthesis with Diffusion Models
  7. 2025MVGBench: Comprehensive Benchmark for Multi-view Generation Models
  8. 2023Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models