A Survey On Text-to-3D Contents Generation In The Wild
Not on the map. Why?
Abstract (arXiv)
3D content creation plays a vital role in various applications, such as gaming, robotics simulation, and virtual reality. However, the process is labor-intensive and time-consuming, requiring skilled designers to invest considerable effort in creating a single 3D asset. To address this challenge, text-to-3D generation technologies have emerged as a promising solution for automating 3D creation. Leveraging the success of large vision language models, these techniques aim to generate 3D content based on textual descriptions. Despite recent advancements in this area, existing solutions still face significant limitations in terms of generation quality and efficiency. In this survey, we conduct an in-depth investigation of the latest text-to-3D creation methods. We provide a comprehensive background on text-to-3D creation, including discussions on datasets employed in training and evaluation metrics used to assess the quality of generated 3D models. Then, we delve into the various 3D representations that serve as the foundation for the 3D generation process. Furthermore, we present a thorough comparison of the rapidly growing literature on generative pipelines, categorizing them into feedforward generators, optimization-based generation, and view reconstruction approaches. By examining the strengths and weaknesses of these methods, we aim to shed light on their respective capabilities and limitations. Lastly, we point out several promising avenues for future research. With this survey, we hope to inspire researchers further to explore the potential of open-vocabulary text-conditioned 3D content creation.
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 89 of its 99 references that are in the map. First posted on arXiv 15 May 2024.
Builds on
89 of 99 references.
References · 89
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
38 more
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
- 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
- 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
- 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
- 2021Variational Diffusion Modelscited by 1,145
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
- 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
- 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
- 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
- 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
- 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
- 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
- 2017Generative Adversarial Networks: An Overviewcited by 588
- 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
- 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 516
- 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
- 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
- 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 449
- 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
- 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
- 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
- 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
- 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
- 2023Text-to-3D using Gaussian Splattingcited by 299
- 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
- 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
- 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 262
- 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
- 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
- 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 225
- 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
- 2023Scalable 3D Captioning with Pretrained Modelscited by 201
- 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 199
50 most cited of 89 All 89 (JSON)
Built on by
Citing papers per year: 2024: 1; 2025: 6; 2026: 6.
13 papers.
Sort:
By area:
- 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 14
- 2024MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D Content Creationcited by 7
- 2026Skin Tokens: A Learned Compact Representation for Unified Autoregressive Riggingcited by 5
- 2025ConsDreamer: Advancing Multi-View Consistency for Zero-Shot Text-to-3D Generationcited by 2
- 2025Towards Fine-Grained Text-to-3D Quality Assessment: A Benchmark and A Two-Stage Rank-Learning Metriccited by 2
- 2025Unifi3D: A Study on 3D Representations for Generation and Reconstruction in a Common Frameworkcited by 1
- 2025FlashMesh: Faster and Better Autoregressive Mesh Synthesis via Structured Speculationcited by 1
- 2026From Idea to Co-Creation: A Planner-Actor-Critic Framework for Agent Augmented 3D Modelingcited by 1
- 2026Clarify Before Executing: A Self-Evolving Agent for Resolving Intent Asymmetry in 3D Tool Orchestrationcited by 1
- 2025Advancing Text-to-3D Generation with Linearized Lookahead Variational Score Distillationcited by 0
- 2026ReConText3D: Replay-based Continual Text-to-3D Generationcited by 0
- 2026From Prompt to Service: An SLM-Based Agent Orchestration Gateway for AI-Driven Virtual Worldscited by 0
1 more
- 2026Clarify Before Executing: A Self-Evolving Agent for Resolving Intent Asymmetry in 3D Tool Orchestration17 Jul 2026
- 20263D-CBM: A Framework for Concept-Based Interpretability in Generative 3D Modeling9 Jun 2026
- 2026From Prompt to Service: An SLM-Based Agent Orchestration Gateway for AI-Driven Virtual Worlds2 Jun 2026
- 2026ReConText3D: Replay-based Continual Text-to-3D Generation15 Apr 2026
- 2026Skin Tokens: A Learned Compact Representation for Unified Autoregressive Rigging4 Feb 2026
- 2026From Idea to Co-Creation: A Planner-Actor-Critic Framework for Agent Augmented 3D Modeling8 Jan 2026
- 2025FlashMesh: Faster and Better Autoregressive Mesh Synthesis via Structured Speculation19 Nov 2025
- 2025Towards Fine-Grained Text-to-3D Quality Assessment: A Benchmark and A Two-Stage Rank-Learning Metric28 Sep 2025
- 2025Unifi3D: A Study on 3D Representations for Generation and Reconstruction in a Common Framework2 Sep 2025
- 2025Advancing Text-to-3D Generation with Linearized Lookahead Variational Score Distillation13 Jul 2025
- 2025ConsDreamer: Advancing Multi-View Consistency for Zero-Shot Text-to-3D Generation3 Apr 2025
- 2025Simulating the Real World: A Unified Survey of Multimodal Generative Models6 Mar 2025
- 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 14
- 2024MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D Content Creationcited by 7
- 2026Skin Tokens: A Learned Compact Representation for Unified Autoregressive Riggingcited by 5
- 2025ConsDreamer: Advancing Multi-View Consistency for Zero-Shot Text-to-3D Generationcited by 2
- 2025Towards Fine-Grained Text-to-3D Quality Assessment: A Benchmark and A Two-Stage Rank-Learning Metriccited by 2
- 2025Unifi3D: A Study on 3D Representations for Generation and Reconstruction in a Common Frameworkcited by 1
- 2025FlashMesh: Faster and Better Autoregressive Mesh Synthesis via Structured Speculationcited by 1
- 2026From Idea to Co-Creation: A Planner-Actor-Critic Framework for Agent Augmented 3D Modelingcited by 1
- 2026Clarify Before Executing: A Self-Evolving Agent for Resolving Intent Asymmetry in 3D Tool Orchestrationcited by 1
- 2026ReConText3D: Replay-based Continual Text-to-3D Generationcited by 0
- 2026From Prompt to Service: An SLM-Based Agent Orchestration Gateway for AI-Driven Virtual Worldscited by 0
- 20263D-CBM: A Framework for Concept-Based Interpretability in Generative 3D Modelingcited by 0
- 2026Clarify Before Executing: A Self-Evolving Agent for Resolving Intent Asymmetry in 3D Tool Orchestration17 Jul 2026
- 20263D-CBM: A Framework for Concept-Based Interpretability in Generative 3D Modeling9 Jun 2026
- 2026From Prompt to Service: An SLM-Based Agent Orchestration Gateway for AI-Driven Virtual Worlds2 Jun 2026
- 2026ReConText3D: Replay-based Continual Text-to-3D Generation15 Apr 2026
- 2026Skin Tokens: A Learned Compact Representation for Unified Autoregressive Rigging4 Feb 2026
- 2026From Idea to Co-Creation: A Planner-Actor-Critic Framework for Agent Augmented 3D Modeling8 Jan 2026
- 2025FlashMesh: Faster and Better Autoregressive Mesh Synthesis via Structured Speculation19 Nov 2025
- 2025Towards Fine-Grained Text-to-3D Quality Assessment: A Benchmark and A Two-Stage Rank-Learning Metric28 Sep 2025
- 2025Unifi3D: A Study on 3D Representations for Generation and Reconstruction in a Common Framework2 Sep 2025
- 2025ConsDreamer: Advancing Multi-View Consistency for Zero-Shot Text-to-3D Generation3 Apr 2025
- 2025Simulating the Real World: A Unified Survey of Multimodal Generative Models6 Mar 2025
- 2024MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D Content Creation26 Nov 2024
Similar, not cited
Close in wording, no citation either way
- 2024Challenges and Opportunities in 3D Content Generation
- 2023ControlDreamer: Blending Geometry and Style in Text-to-3D
- 2025Recent Advances in 3D Object and Scene Generation: A Survey
- 2023Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields
- 2026Co-generation of Layout and Shape from Text via Autoregressive 3D Diffusion
- 2023X-Dreamer: Creating High-quality 3D Content by Bridging the Domain Gap Between Text-to-2D and Text-to-3D Generation
- 2025Sel3DCraft: Interactive Visual Prompts for User-Friendly Text-to-3D Generation
- 2025DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization