mlsanityData to 7 Oct 2026

Zero-1-to-3: Zero-shot One Image to 3D Object

arXiv 2303.11328 ↗First posted 20 Mar 2023cs.CV cs.GR cs.ROcited by 1,483

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We introduce Zero-1-to-3, a framework for changing the camera viewpoint of an object given just a single RGB image. To perform novel view synthesis in this under-constrained setting, we capitalize on the geometric priors that large-scale diffusion models learn about natural images. Our conditional diffusion model uses a synthetic dataset to learn controls of the relative camera viewpoint, which allow new images to be generated of the same object under a specified camera transformation. Even though it is trained on a synthetic dataset, our model retains a strong zero-shot generalization ability to out-of-distribution datasets as well as in-the-wild images, including impressionist paintings. Our viewpoint-conditioned diffusion approach can further be used for the task of 3D reconstruction from a single image. Qualitative and quantitative experiments show that our method significantly outperforms state-of-the-art single-view 3D reconstruction and novel view synthesis models by leveraging Internet-scale pre-training.

Key facts. Cited by 1,483 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 58 of its 73 references that are in the map. First posted on arXiv 20 Mar 2023.

Builds on

58 of 73 references.

Closest ancestors · 7

Same direction, newest first

  1. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  2. 2022NeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priors6 Dec
  3. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  4. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views29 Nov
  5. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  6. 2022Novel View Synthesis with Diffusion Models6 Oct
  7. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 51

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2022Classifier-Free Diffusion Guidancecited by 5,717
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  12. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
38 more
  1. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  2. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  3. 2022Video Diffusion Modelscited by 2,280
  4. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  5. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  6. 2021Vision Transformers for Dense Predictioncited by 1,789
  7. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  8. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  9. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  10. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  11. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  12. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  13. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  14. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  15. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  16. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  17. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  18. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  19. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  20. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  21. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  22. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  23. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  24. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 332
  25. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  26. 2018Learning 3D Human Dynamics from Videocited by 315
  27. 20163D Menagerie: Modeling the 3D shape and pose of animalscited by 310
  28. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  29. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  30. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  31. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  32. 2017MarrNet: 3D Shape Reconstruction via 2.5D Sketchescited by 108
  33. 2019Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"cited by 106
  34. 2023Multiview Compressive Coding for 3D Reconstructioncited by 71
  35. 2022Multi-View Mesh Reconstruction with Neural Deferred Shadingcited by 44
  36. 2022RTMV: A Ray-Traced Multi-View Synthetic Dataset for Novel View Synthesiscited by 33
  37. 2022Revealing Occlusions with 4D Neural Fieldscited by 17
  38. 2022Planes vs. Chairs: Category-guided 3D shape learning without any 3D cuescited by 9

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 222; 2024: 560; 2025: 439; 2026: 262.

1,483 papers.

Sort:

By area:

  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  2. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  3. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  4. 2025Qwen-Image Technical Reportcited by 973
  5. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  6. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  7. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  8. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  9. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  10. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  11. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 707
  12. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
38 more
  1. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  2. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  3. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  4. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 516
  5. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 444
  6. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 425
  7. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  8. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  9. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
  10. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  11. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  12. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 342
  13. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  14. 2023ReconFusion: 3D Reconstruction with Diffusion Priorscited by 317
  15. 2023Text-to-3D using Gaussian Splattingcited by 299
  16. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 298
  17. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 296
  18. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  19. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  20. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Imagecited by 283
  21. 2023FSGS: Real-Time Few-shot View Synthesis using Gaussian Splattingcited by 278
  22. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
  23. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  24. 2024Navigation World Modelscited by 262
  25. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 257
  26. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
  27. 2024DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalizationcited by 256
  28. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  29. 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 227
  30. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 214
  31. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 213
  32. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
  33. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
  34. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  35. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 191
  36. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 184
  37. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 183
  38. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179

50 most cited of 1,483 All 1,483 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025DT-NVS: Diffusion Transformers for Novel View Synthesis
  2. 2024FSViewFusion: Few-Shots View Generation of Novel Objects
  3. 2024Novel View Synthesis with Pixel-Space Diffusion Models
  4. 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction
  5. 2025Zero-P-to-3: Zero-Shot Partial-View Images to 3D Object
  6. 2024Ctrl123: Consistent Novel View Synthesis via Closed-Loop Transcription
  7. 2026One-Shot Novel View and Pose Human Image Synthesis via 3D Prior Guided Diffusion Model
  8. 2023Enhancing Diffusion Models with 3D Perspective Geometry Constraints