mlsanityData to 7 Oct 2026

Text-to-3D Shape Generation

arXiv 2403.13289 ↗First posted 20 Mar 2024cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

Recent years have seen an explosion of work and interest in text-to-3D shape generation. Much of the progress is driven by advances in 3D representations, large-scale pretraining and representation learning for text and image data enabling generative AI models, and differentiable rendering. Computational systems that can perform text-to-3D shape generation have captivated the popular imagination as they enable non-expert users to easily create 3D content directly from text. However, there are still many limitations and challenges remaining in this problem space. In this state-of-the-art report, we provide a survey of the underlying technology and methods enabling text-to-3D shape generation to summarize the background literature. We then derive a systematic categorization of recent work on text-to-3D shape generation based on the type of supervision data required. Finally, we discuss limitations of the existing categories of methods, and delineate promising directions for future work.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 129 of its 151 references that are in the map. First posted on arXiv 20 Mar 2024.

Builds on

129 of 151 references.

References · 129

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  12. 2022Classifier-Free Diffusion Guidancecited by 5,717
38 more
  1. 2017Neural Discrete Representation Learningcited by 4,650
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  4. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  6. 2016Improved Techniques for Training GANscited by 2,938
  7. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  8. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  9. 2017Improved Training of Wasserstein GANscited by 2,223
  10. 2014Conditional Generative Adversarial Netscited by 2,152
  11. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  12. 2016Density estimation using Real NVPcited by 2,029
  13. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  14. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  15. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  16. 2021Vision Transformers for Dense Predictioncited by 1,789
  17. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  18. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  19. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  20. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  21. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  22. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  23. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  24. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  25. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  26. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  27. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  28. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  29. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  30. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  31. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  32. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  33. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  34. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  35. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  36. 2021Neural Fields in Visual Computing and Beyondcited by 539
  37. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 516
  38. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513

50 most cited of 129 All 129 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 6; 2026: 2.

13 papers.

Sort:

By area:

  1. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 41
  2. 2024An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusioncited by 34
  3. 2025NuiScene: Exploring Efficient Generation of Unbounded Outdoor Scenescited by 19
  4. 2024A Survey of Sustainability in Large Language Models: Applications, Economics, and Challengescited by 18
  5. 2024STAR: Skeleton-aware Text-based 4D Avatar Generation with In-Network Motion Retargetingcited by 13
  6. 2024A Survey on Computational Solutions for Reconstructing Complete Objects by Reassembling Their Fractured Partscited by 8
  7. 2025LLM-to-Phy3D: Physically Conform Online 3D Object Generation with LLMscited by 4
  8. 2026Deep Sketch-Based 3D Modeling: A Surveycited by 3
  9. 2025Boosting 3D Liver Shape Datasets with Diffusion Models and Implicit Neural Representationscited by 1
  10. 2025GeoTexBuild: 3D Building Model Generation from Map Footprintscited by 0
  11. 20253D Shape Generation: A Surveycited by 0
  12. 2025Toward Using Machine Learning as a Shape Quality Metric for Liver Point Cloud Generationcited by 0
1 more
  1. 2026A Research Prototype for Closed-Loop Generative Design of Customized Foot Orthoses via Semantic-Physics Alignmentcited by 0