mlsanityData to 7 Oct 2026

A Survey On Text-to-3D Contents Generation In The Wild

arXiv 2405.09431 ↗First posted 15 May 2024cs.CV cs.GRcited by 13

Not on the map. Why?

Abstract (arXiv)

3D content creation plays a vital role in various applications, such as gaming, robotics simulation, and virtual reality. However, the process is labor-intensive and time-consuming, requiring skilled designers to invest considerable effort in creating a single 3D asset. To address this challenge, text-to-3D generation technologies have emerged as a promising solution for automating 3D creation. Leveraging the success of large vision language models, these techniques aim to generate 3D content based on textual descriptions. Despite recent advancements in this area, existing solutions still face significant limitations in terms of generation quality and efficiency. In this survey, we conduct an in-depth investigation of the latest text-to-3D creation methods. We provide a comprehensive background on text-to-3D creation, including discussions on datasets employed in training and evaluation metrics used to assess the quality of generated 3D models. Then, we delve into the various 3D representations that serve as the foundation for the 3D generation process. Furthermore, we present a thorough comparison of the rapidly growing literature on generative pipelines, categorizing them into feedforward generators, optimization-based generation, and view reconstruction approaches. By examining the strengths and weaknesses of these methods, we aim to shed light on their respective capabilities and limitations. Lastly, we point out several promising avenues for future research. With this survey, we hope to inspire researchers further to explore the potential of open-vocabulary text-conditioned 3D content creation.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 89 of its 99 references that are in the map. First posted on arXiv 15 May 2024.

Builds on

89 of 99 references.

References · 89

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  11. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  12. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
38 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  2. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  4. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  5. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  6. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  7. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  8. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  9. 2021Variational Diffusion Modelscited by 1,145
  10. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  11. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  12. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  13. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  14. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  15. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  16. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  17. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  18. 2017Generative Adversarial Networks: An Overviewcited by 588
  19. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  20. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 516
  21. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  22. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  23. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 449
  24. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  25. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  26. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  27. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  28. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
  29. 2023Text-to-3D using Gaussian Splattingcited by 299
  30. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  31. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
  32. 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 262
  33. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  34. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  35. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 225
  36. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  37. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  38. 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 199

50 most cited of 89 All 89 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 6.

13 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2024Challenges and Opportunities in 3D Content Generation
  2. 2023ControlDreamer: Blending Geometry and Style in Text-to-3D
  3. 2025Recent Advances in 3D Object and Scene Generation: A Survey
  4. 2023Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields
  5. 2026Co-generation of Layout and Shape from Text via Autoregressive 3D Diffusion
  6. 2023X-Dreamer: Creating High-quality 3D Content by Bridging the Domain Gap Between Text-to-2D and Text-to-3D Generation
  7. 2025Sel3DCraft: Interactive Visual Prompts for User-Friendly Text-to-3D Generation
  8. 2025DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization