mlsanityData to 7 Oct 2026

Generative AI meets 3D: A Survey on Text-to-3D in AIGC Era

arXiv 2305.06131 ↗First posted 10 May 2023cs.CVcited by 61

Not on the map. Why?

Abstract (arXiv)

Generative AI has made significant progress in recent years, with text-guided content generation being the most practical as it facilitates interaction between human instructions and AI-generated content (AIGC). Thanks to advancements in text-to-image and 3D modeling technologies, like neural radiance field (NeRF), text-to-3D has emerged as a nascent yet highly active research field. Our work conducts a comprehensive survey on this topic and follows up on subsequent research progress in the overall field, aiming to help readers interested in this direction quickly catch up with its rapid development. First, we introduce 3D data representations, including both Structured and non-Structured data. Building on this pre-requisite, we introduce various core technologies to achieve satisfactory text-to-3D results. Additionally, we present mainstream baselines and research directions in recent text-to-3D technology, including fidelity, efficiency, consistency, controllability, diversity, and applicability. Furthermore, we summarize the usage of text-to-3D technology in various applications, including avatar generation, texture generation, scene generation and 3D editing. Finally, we discuss the agenda for the future development of text-to-3D.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). Builds on 98 of its 226 references that are in the map. First posted on arXiv 10 May 2023.

Builds on

98 of 226 references.

References · 98

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  12. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
38 more
  1. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  3. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  6. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  7. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  8. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  9. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  10. 2021Vision Transformers for Dense Predictioncited by 1,789
  11. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  12. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  13. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  14. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  15. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  16. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  17. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  18. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  19. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  20. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  21. 2019Deep Learning for 3D Point Clouds: A Surveycited by 557
  22. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  23. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  24. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  25. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  26. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 482
  27. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  28. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  29. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 406
  30. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  31. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  32. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  33. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 303
  34. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  35. 20223D Neural Field Generation using Triplane Diffusioncited by 274
  36. 2023Text-to-image Diffusion Models in Generative AI: A Surveycited by 264
  37. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  38. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235

50 most cited of 98 All 98 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 26; 2025: 20; 2026: 5.

61 papers.

Sort:

By area:

  1. 2023A Survey on Video Diffusion Modelscited by 203
  2. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  3. 2024MeshXL: Neural Coordinate Field for Generative 3D Foundation Modelscited by 100
  4. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 82
  5. 2024Advances in 3D Generation: A Surveycited by 62
  6. 2024Sora as a World Model? A Complete Survey on Text-to-Video Generationcited by 53
  7. 2023HeadSculpt: Crafting 3D Head Avatars with Textcited by 48
  8. 2024REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignmentcited by 45
  9. 2025TreeMeshGPT: Artistic Mesh Generation with Autoregressive Tree Sequencingcited by 38
  10. 2024HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splattingcited by 33
  11. 2024Make-Your-3D: Fast and Consistent Subject-Driven 3D Content Generationcited by 32
  12. 2023Robustness of SAM: Segment Anything Under Corruptions and Beyondcited by 30
38 more
  1. 2024A Comprehensive Survey on 3D Content Generationcited by 28
  2. 2025DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimizationcited by 27
  3. 20253D Scene Generation: A Surveycited by 24
  4. 2024BrightDreamer: Generic 3D Gaussian Generative Framework for Fast Text-to-3D Synthesiscited by 20
  5. 2024Diffusion-Based Visual Art Creation: A Survey and New Perspectivescited by 17
  6. 20244Dynamic: Text-to-4D Generation with Hybrid Priorscited by 15
  7. 2023Robustness of Segment Anything Model (SAM) for Autonomous Driving in Adverse Weather Conditionscited by 14
  8. 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 14
  9. 2023Black-box Targeted Adversarial Attack on Segment Anything (SAM)cited by 13
  10. 2024Text-to-3D Shape Generationcited by 13
  11. 2024A Survey On Text-to-3D Contents Generation In The Wildcited by 13
  12. 2024A Survey on 3D Human Avatar Modeling -- From Reconstruction to Generationcited by 13
  13. 2024STAR: Skeleton-aware Text-based 4D Avatar Generation with In-Network Motion Retargetingcited by 13
  14. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  15. 2025BrepGPT: Autoregressive B-rep Generation with Voronoi Half-Patchcited by 12
  16. 2023Understanding Segment Anything Model: SAM is Biased Towards Texture Rather than Shapecited by 10
  17. 2024Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Reviewcited by 9
  18. 2024DreamView: Injecting View-specific Text Guidance into Text-to-3D Generationcited by 9
  19. 2023Segment Anything Meets Semantic Communicationcited by 8
  20. 2024Make-It-Vivid: Dressing Your Animatable Biped Cartoon Characters from Textcited by 7
  21. 2024MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D Content Creationcited by 7
  22. 2023A Survey of Generative AI for Intelligent Transportation Systems: Road Transportation Perspectivecited by 6
  23. 2025DiMeR: Disentangled Mesh Reconstruction Modelcited by 6
  24. 2025Is Single-View Mesh Reconstruction Ready for Robotics?cited by 6
  25. 2024MagicNaming: Consistent Identity Generation by Finding a "Name Space" in T2I Diffusion Modelscited by 5
  26. 2025SIR-DIFF: Sparse Image Sets Restoration with Multi-View Diffusion Modelcited by 5
  27. 2023SAM Meets UAP: Attacking Segment Anything Model With Universal Adversarial Perturbationcited by 4
  28. 2024Speech to Reality: On-Demand Production using Natural Language, 3D Generative AI, and Discrete Robotic Assemblycited by 4
  29. 2025GaussianSeal: Rooting Adaptive Watermarks for 3D Gaussian Generation Modelcited by 4
  30. 2025A Comprehensive Survey on Visual Concept Mining in Text-to-image Diffusion Modelscited by 4
  31. 2025PlantDreamer: Achieving Realistic 3D Plant Models with Diffusion-Guided Gaussian Splattingcited by 3
  32. 2026Deep Sketch-Based 3D Modeling: A Surveycited by 3
  33. 2024Toward Scene Graph and Layout Guided Complex 3D Scene Generationcited by 2
  34. 2025DreamAnywhere: Object-Centric Panoramic 3D Scene Generationcited by 2
  35. 2025EigenActor: Variant Body-Object Interaction Generation Evolved from Invariant Action Basis Reasoningcited by 1
  36. 2025From Air to Wear: Personalized 3D Digital Fashion with AR/VR Immersive 3D Sketchingcited by 1
  37. 2025Text to Robotic Assembly of Multi Component Objects using 3D Generative AI and Vision Language Modelscited by 1
  38. 2025ITS3D: Inference-Time Scaling for Text-Guided 3D Diffusion Modelscited by 1

50 most cited of 61 All 61 (JSON)