mlsanityData to 7 Oct 2026

Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors

arXiv 2306.17843 ↗First posted 30 Jun 2023cs.CVcited by 395

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present Magic123, a two-stage coarse-to-fine approach for high-quality, textured 3D meshes generation from a single unposed image in the wild using both2D and 3D priors. In the first stage, we optimize a neural radiance field to produce a coarse geometry. In the second stage, we adopt a memory-efficient differentiable mesh representation to yield a high-resolution mesh with a visually appealing texture. In both stages, the 3D content is learned through reference view supervision and novel views guided by a combination of 2D and 3D diffusion priors. We introduce a single trade-off parameter between the 2D and 3D priors to control exploration (more imaginative) and exploitation (more precise) of the generated geometry. Additionally, we employ textual inversion and monocular depth regularization to encourage consistent appearances across views and to prevent degenerate solutions, respectively. Magic123 demonstrates a significant improvement over previous image-to-3D techniques, as validated through extensive experiments on synthetic benchmarks and diverse real-world images. Our code, models, and generated 3D assets are available at https://github.com/guochengqian/Magic123.

Key facts. Cited by 395 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 88 of its 104 references that are in the map. First posted on arXiv 30 Jun 2023.

Builds on

88 of 104 references.

Closest ancestors · 15

Same direction, newest first

  1. 2023Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusion20 Apr
  2. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
  3. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  4. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Models20 Mar
  5. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  6. 2023SKED: Sketch-guided Text-based 3D Editing19 Mar
  7. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation14 Mar
  8. 2023TEXTure: Text-Guided Texturing of 3D Shapes3 Feb
  9. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models28 Dec
  10. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  11. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  12. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views29 Nov
3 more
  1. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  2. 2022Novel View Synthesis with Diffusion Models6 Oct
  3. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 73

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  11. 2022Generative Adversarial Networkscited by 5,603
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,276
38 more
  1. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  2. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  3. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  4. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  5. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  6. 2021Vision Transformers for Dense Predictioncited by 1,789
  7. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  8. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  9. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  10. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  11. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  12. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  13. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  14. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  15. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  16. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  17. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  18. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  19. 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 569
  20. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  21. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  22. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  23. 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 467
  24. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  25. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  26. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  27. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  28. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  29. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  30. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  31. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  32. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
  33. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  34. 2018DeepMVS: Learning Multi-view Stereopsiscited by 277
  35. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 267
  36. 2019Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inferencecited by 252
  37. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  38. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 84; 2024: 197; 2025: 81; 2026: 33.

395 papers.

Sort:

By area:

  1. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  2. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  3. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  4. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  5. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 444
  6. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 425
  7. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  8. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  9. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  10. 2023Text-to-3D using Gaussian Splattingcited by 299
  11. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  12. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
38 more
  1. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
  2. 2024DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalizationcited by 256
  3. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  4. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
  5. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 214
  6. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 213
  7. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 191
  8. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 184
  9. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 175
  10. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  11. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  12. 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Modelscited by 160
  13. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Priorcited by 157
  14. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  15. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  16. 2024Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Imagecited by 132
  17. 2023Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Modelscited by 122
  18. 2024CraftsMan3D: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refinercited by 120
  19. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 118
  20. 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
  21. 2023GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondencecited by 113
  22. 2023Animate124: Animating One Image to 4D Dynamic Scenecited by 113
  23. 2023TADA! Text to Animatable Digital Avatarscited by 109
  24. 2023ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Imagecited by 109
  25. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusioncited by 109
  26. 2023TeCH: Text-guided Reconstruction of Lifelike Clothed Humanscited by 104
  27. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
  28. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 99
  29. 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 98
  30. 2024MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstructioncited by 94
  31. 2023WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generationcited by 93
  32. 2024GES: Generalized Exponential Splatting for Efficient Radiance Field Renderingcited by 92
  33. 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 88
  34. 2024Meshtron: High-Fidelity, Artist-Like 3D Mesh Generation at Scalecited by 87
  35. 2024IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generationcited by 84
  36. 2024EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generationcited by 84
  37. 2023A Unified Approach for Text- and Image-guided 4D Scene Generationcited by 82
  38. 2023SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstructioncited by 77

50 most cited of 395 All 395 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior
  2. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization
  3. 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers
  4. 2025Recent Advances in 3D Object and Scene Generation: A Survey
  5. 20233D-aware Image Generation using 2D Diffusion Models
  6. 2023Breathing New Life into 3D Assets with Generative Repainting
  7. 2025Self-Evolving 3D Scene Generation from a Single Image
  8. 2024MagicMan: Generative Novel View Synthesis of Humans with 3D-Aware Diffusion and Iterative Refinement