mlsanityData to 14 Oct 2026

Magic3D: High-Resolution Text-to-3D Content Creation

arXiv 2211.10440 ↗First posted 18 Nov 2022cs.CV cs.GR cs.LGcited by 1,290

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

DreamFusion has recently demonstrated the utility of a pre-trained text-to-image diffusion model to optimize Neural Radiance Fields (NeRF), achieving remarkable text-to-3D synthesis results. However, the method has two inherent limitations: (a) extremely slow optimization of NeRF and (b) low-resolution image space supervision on NeRF, leading to low-quality 3D models with a long processing time. In this paper, we address these limitations by utilizing a two-stage optimization framework. First, we obtain a coarse model using a low-resolution diffusion prior and accelerate with a sparse 3D hash grid structure. Using the coarse representation as the initialization, we further optimize a textured 3D mesh model with an efficient differentiable renderer interacting with a high-resolution latent diffusion model. Our method, dubbed Magic3D, can create high quality 3D mesh models in 40 minutes, which is 2x faster than DreamFusion (reportedly taking 1.5 hours on average), while also achieving higher resolution. User studies show 61.7% raters to prefer our approach over DreamFusion. Together with the image-conditioned generation capabilities, we provide users with new ways to control 3D synthesis, opening up new avenues to various creative applications.

Key facts. Cited by 1,290 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 48 of its 56 references that are in the map. First posted on arXiv 18 Nov 2022.

Builds on

48 of 56 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  2. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  3. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 45

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 2022Classifier-Free Diffusion Guidancecited by 5,847
  9. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  10. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  12. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
33 more
  1. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  2. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  3. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  4. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  5. 2020Neural Sparse Voxel Fieldscited by 959
  6. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  7. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  8. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  9. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 624
  10. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  11. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  12. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  13. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 488
  14. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  15. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 460
  16. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  17. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 441
  18. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  19. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 366
  20. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  21. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  22. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  23. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  24. 2022VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Gridscited by 133
  25. 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
  26. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  27. 2021GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worldscited by 98
  28. 2020Learning Deformable Tetrahedral Meshes for 3D Reconstructioncited by 85
  29. 2017Improved Adversarial Systems for 3D Object Generation and Reconstructioncited by 66
  30. 2019HoloGAN: Unsupervised learning of 3D representations from natural imagescited by 59
  31. 2019StructureNet: Hierarchical Graph Networks for 3D Shape Generationcited by 59
  32. 2021Octree Transformer: Autoregressive 3D Shape Generation on Hierarchically Structured Sequencescited by 39
  33. 2020Inverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Datacited by 30

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 6; 2023: 355; 2024: 485; 2025: 270; 2026: 174.

1,290 papers.

Sort:

By area:

  1. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  2. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  3. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
  4. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
  5. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
  6. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
  7. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  8. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 699
  9. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
  10. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  11. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
  12. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 587
38 more
  1. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 560
  2. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 484
  3. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  4. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 455
  5. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 433
  6. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
  7. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 402
  8. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 397
  9. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 385
  10. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 378
  11. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  12. 2023DeepCache: Accelerating Diffusion Models for Freecited by 359
  13. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 356
  14. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 343
  15. 2023GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splattingcited by 335
  16. 2023ReconFusion: 3D Reconstruction with Diffusion Priorscited by 318
  17. 2023SeeSR: Towards Semantics-Aware Real-World Image Super-Resolutioncited by 315
  18. 2023Your Diffusion Model is Secretly a Zero-Shot Classifiercited by 306
  19. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 304
  20. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 301
  21. 2023Text-to-3D using Gaussian Splattingcited by 299
  22. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
  23. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  24. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  25. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 279
  26. 2023FSGS: Real-Time Few-shot View Synthesis using Gaussian Splattingcited by 279
  27. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 274
  28. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 268
  29. 2023Text-to-image Diffusion Models in Generative AI: A Surveycited by 265
  30. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 261
  31. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 259
  32. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 258
  33. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 257
  34. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  35. 2024Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Modelscited by 254
  36. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  37. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  38. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Modelscited by 236

50 most cited of 1,290 All 1,290 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models
  2. 2023Breathing New Life into 3D Assets with Generative Repainting
  3. 20223DDesigner: Towards Photorealistic 3D Object Generation and Editing with Text-guided Diffusion Models
  4. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures
  5. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusion
  6. 2025DT-NeRF: A Diffusion and Transformer-Based Optimization Approach for Neural Radiance Fields in 3D Reconstruction
  7. 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction
  8. 2022NeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priors