mlsanityData to 7 Oct 2026

Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior

arXiv 2303.14184 ↗First posted 24 Mar 2023cs.CVcited by 341

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

In this work, we investigate the problem of creating high-fidelity 3D content from only a single image. This is inherently challenging: it essentially involves estimating the underlying 3D geometry while simultaneously hallucinating unseen textures. To address this challenge, we leverage prior knowledge from a well-trained 2D diffusion model to act as 3D-aware supervision for 3D creation. Our approach, Make-It-3D, employs a two-stage optimization pipeline: the first stage optimizes a neural radiance field by incorporating constraints from the reference image at the frontal view and diffusion prior at novel views; the second stage transforms the coarse model into textured point clouds and further elevates the realism with diffusion prior while leveraging the high-quality textures from the reference image. Extensive experiments demonstrate that our method outperforms prior works by a large margin, resulting in faithful reconstructions and impressive visual quality. Our method presents the first attempt to achieve high-quality 3D creation from a single image for general objects and enables various applications such as text-to-3D creation and texture editing.

Key facts. Cited by 341 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 54 of its 64 references that are in the map. First posted on arXiv 24 Mar 2023.

Builds on

54 of 64 references.

Closest ancestors · 9

Same direction, newest first

  1. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  2. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion12 Dec
  3. 2022NeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priors6 Dec
  4. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  5. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views29 Nov
  6. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  7. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation17 Nov
  8. 2022Novel View Synthesis with Diffusion Models6 Oct
  9. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 45

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2017Neural Discrete Representation Learningcited by 4,650
  9. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  10. 2021Vision Transformers for Dense Predictioncited by 1,789
  11. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  12. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
33 more
  1. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  2. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  3. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  4. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  5. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  6. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  7. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  8. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  9. 2019Neural Point-Based Graphicscited by 326
  10. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  11. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  12. 20203D Photography using Context-aware Layered Depth Inpaintingcited by 206
  13. 2019Pushing the Boundaries of View Extrapolation with Multiplane Imagescited by 191
  14. 2020GRF: Learning a General Radiance Field for 3D Representation and Renderingcited by 183
  15. 2016View Synthesis by Appearance Flowcited by 172
  16. 2022SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Imagecited by 165
  17. 2022Pretraining is All You Need for Image-to-Image Translationcited by 160
  18. 2018The Contextual Loss for Image Transformation with Non-Aligned Datacited by 135
  19. 2021MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesiscited by 133
  20. 20193D Ken Burns Effect from a Single Imagecited by 128
  21. 2018Layer-structured 3D Scene Inference via View Synthesiscited by 82
  22. 2021PixelSynth: Generating a 3D-Consistent Experience from a Single Imagecited by 82
  23. 2022High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimizationcited by 65
  24. 2020Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANscited by 63
  25. 2017Learning to Synthesize a 4D RGBD Light Field from a Single Imagecited by 60
  26. 2017Transformation-Grounded Image Generation Network for Novel 3D View Synthesiscited by 59
  27. 2022Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Imagescited by 59
  28. 2022Understanding Pure CLIP Guidance for Voxel Grid NeRF Modelscited by 56
  29. 2021SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpaintingcited by 48
  30. 2019Transformable Bottleneck Networkscited by 23
  31. 2018Geometry-aware Deep Network for Single-Image Novel View Synthesiscited by 19
  32. 2016Novel Views of Objects from a Single Imagecited by 12
  33. 2018Fast View Synthesis with Deep Stereo Visioncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 95; 2024: 153; 2025: 60; 2026: 33.

341 papers.

Sort:

By area:

  1. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  2. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  3. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  4. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  5. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  6. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  7. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  8. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 425
  9. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  10. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
  11. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 298
  12. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 296
38 more
  1. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  2. 2023FSGS: Real-Time Few-shot View Synthesis using Gaussian Splattingcited by 278
  3. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  4. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 257
  5. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
  6. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  7. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 191
  8. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 184
  9. 2024Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformerscited by 169
  10. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  11. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Priorcited by 157
  12. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  13. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  14. 2024MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformerscited by 140
  15. 2023Re-imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D, alleviate Janus problem and Beyondcited by 139
  16. 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 138
  17. 2023Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Videocited by 124
  18. 2023Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Modelscited by 122
  19. 2024Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Controlcited by 117
  20. 2023Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusioncited by 114
  21. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 113
  22. 2023HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidancecited by 109
  23. 2023SyncDiffusion: Coherent Montage via Synchronized Joint Diffusionscited by 108
  24. 2023TeCH: Text-guided Reconstruction of Lifelike Clothed Humanscited by 104
  25. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
  26. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 99
  27. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
  28. 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 88
  29. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  30. 2024IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generationcited by 84
  31. 2024EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generationcited by 84
  32. 2023A Unified Approach for Text- and Image-guided 4D Scene Generationcited by 82
  33. 2023DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generationcited by 81
  34. 2023Consistent Video-to-Video Transfer Using Synthetic Datasetcited by 79
  35. 2023SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstructioncited by 77
  36. 2024TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And Image-Promptscited by 76
  37. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
  38. 2023SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusioncited by 72

50 most cited of 341 All 341 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior
  2. 2023Neural Point Cloud Diffusion for Disentangled 3D Shape and Appearance Generation
  3. 2023Breathing New Life into 3D Assets with Generative Repainting
  4. 2023Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shape
  5. 2023Single Mesh Diffusion Models with Field Latents for Texture Generation
  6. 2024Gen-3Diffusion: Realistic Image-to-3D Generation via 2D & 3D Diffusion Synergy
  7. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors
  8. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models