mlsanityData to 14 Oct 2026

Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Data

arXiv 2306.07881 ↗First posted 13 Jun 2023cs.CVcited by 113

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present Viewset Diffusion, a diffusion-based generator that outputs 3D objects while only using multi-view 2D data for supervision. We note that there exists a one-to-one mapping between viewsets, i.e., collections of several 2D views of an object, and 3D models. Hence, we train a diffusion model to generate viewsets, but design the neural network generator to reconstruct internally corresponding 3D models, thus generating those too. We fit a diffusion model to a large number of viewsets for a given category of objects. The resulting generator can be conditioned on zero, one or more input views. Conditioned on a single view, it performs 3D reconstruction accounting for the ambiguity of the task and allowing to sample multiple solutions compatible with the input. The model performs reconstruction efficiently, in a feed-forward manner, and is trained using only rendering losses using as few as three views per viewset. Project page: szymanowiczs.github.io/viewset-diffusion.

Key facts. Cited by 113 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 56 of its 60 references that are in the map. First posted on arXiv 13 Jun 2023.

Builds on

56 of 60 references.

Closest ancestors · 13

Same direction, newest first

  1. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
  2. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Images29 Mar
  3. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  4. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generation9 Mar
  5. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion12 Dec
  6. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusion2 Dec
  7. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  8. 20223D Neural Field Generation using Triplane Diffusion30 Nov
  9. 20223DDesigner: Towards Photorealistic 3D Object Generation and Editing with Text-guided Diffusion Models25 Nov
  10. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation17 Nov
  11. 2022Novel View Synthesis with Diffusion Models6 Oct
  12. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
1 more
  1. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  8. 2020Denoising Diffusion Implicit Modelscited by 9,781
  9. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  10. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  11. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  12. 2018Group Normalizationcited by 1,524
31 more
  1. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  2. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 964
  3. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  4. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
  5. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  6. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 588
  7. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  8. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 460
  9. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 385
  10. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 332
  11. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  12. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 306
  13. 2023Efficient Diffusion Training via Min-SNR Weighting Strategycited by 246
  14. 2016Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesiscited by 219
  15. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  16. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  17. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 180
  18. 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
  19. 2022SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Imagecited by 165
  20. 2022Generalizable Patch-Based Neural Renderingcited by 122
  21. 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 112
  22. 2021Light Field Neural Renderingcited by 108
  23. 2016Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervisioncited by 106
  24. 2022Vision Transformer for NeRF-Based View Synthesis from a Single Input Imagecited by 104
  25. 2021ShaRF: Shape-conditioned Radiance Fields from a Single Viewcited by 92
  26. 2018Learning Shape Priors for Single-View 3D Completion and Reconstructioncited by 68
  27. 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
  28. 2021DOVE: Learning Deformable 3D Objects by Watching Videoscited by 61
  29. 2022AutoRF: Learning 3D Object Radiance Fields from Single View Observationscited by 59
  30. 2021Fast and Explicit Neural View Synthesiscited by 28
  31. 2017Weakly supervised 3D Reconstruction with Adversarial Constraintcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 24; 2024: 59; 2025: 20; 2026: 10.

113 papers.

Sort:

By area:

  1. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
  2. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
  3. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 684
  4. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
  5. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
  6. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 364
  7. 2023ReconFusion: 3D Reconstruction with Diffusion Priorscited by 318
  8. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 301
  9. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 257
  10. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 214
  11. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 189
  12. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
38 more
  1. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  2. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 154
  3. 2023State of the Art on Diffusion Models for Visual Computingcited by 127
  4. 2024latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstructioncited by 118
  5. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 116
  6. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 100
  7. 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 89
  8. 2024IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generationcited by 84
  9. 2023Free3D: Consistent Novel View Synthesis without 3D Representationcited by 72
  10. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  11. 2024SPAD : Spatially Aware Multiview Diffuserscited by 62
  12. 2025Bolt3D: Generating 3D Scenes in Secondscited by 59
  13. 2024ViewDiff: 3D-Consistent Image Generation with Text-to-Image Modelscited by 57
  14. 2024VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Modelscited by 56
  15. 2024Part123: Part-aware 3D Reconstruction from a Single-view Imagecited by 55
  16. 2025Amodal3R: Amodal 3D Reconstruction from Occluded 2D Imagescited by 54
  17. 2025DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generationcited by 50
  18. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Modelcited by 45
  19. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 44
  20. 2024Score Distillation via Reparametrized DDIMcited by 41
  21. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 41
  22. 2023VolumeDiffusion: Flexible Text-to-3D Generation with Efficient Volumetric Encodercited by 39
  23. 2024MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generationcited by 35
  24. 2024Tailor3D: Customized 3D Assets Editing and Generation with Dual-Side Imagescited by 35
  25. 2024SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesiscited by 34
  26. 2024Baking Gaussian Splatting into Diffusion Denoiser for Fast and Scalable Single-stage Image-to-3D Generation and Reconstructioncited by 32
  27. 2025SynCity: Training-Free Generation of 3D Worldscited by 31
  28. 2023HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3Dcited by 28
  29. 2024Hi3D: Pursuing High-Resolution Image-to-3D Generation with Video Diffusion Modelscited by 27
  30. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 27
  31. 2023Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repaintingcited by 25
  32. 2025DSO: Aligning 3D Generators with Simulation Feedback for Physical Soundnesscited by 25
  33. 2024Template-Free Single-View 3D Human Digitalization with Diffusion-Guided LRMcited by 23
  34. 2024RelitLRM: Generative Relightable Radiance for Large Reconstruction Modelscited by 23
  35. 2024Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabricationcited by 20
  36. 2023NViST: In the Wild New View Synthesis from a Single Image with Transformerscited by 19
  37. 2025UnCommon Objects in 3Dcited by 17
  38. 2023Efficient-NeRF2NeRF: Streamlining Text-Driven 3D Editing with Multiview Correspondence-Enhanced Diffusion Modelscited by 16

50 most cited of 113 All 113 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Gen-3Diffusion: Realistic Image-to-3D Generation via 2D & 3D Diffusion Synergy
  2. 2023Generating Images with 3D Annotations Using Diffusion Models
  3. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model
  4. 20233D-aware Image Generation using 2D Diffusion Models
  5. 20243D-Adapter: Geometry-Consistent Multi-View Diffusion for High-Quality 3D Generation
  6. 2025Distilling Multi-view Diffusion Models into 3D Generators
  7. 2024Magic-Boost: Boost 3D Generation with Multi-View Conditioned Diffusion
  8. 2024Ouroboros3D: Image-to-3D Generation via 3D-aware Recursive Diffusion