mlsanityData to 7 Oct 2026

Towards Implicit Text-Guided 3D Shape Generation

arXiv 2203.14622 ↗First posted 28 Mar 2022cs.CVcited by 68

Not on the map. Why?

Abstract (arXiv)

In this work, we explore the challenging task of generating 3D shapes from text. Beyond the existing works, we propose a new approach for text-guided 3D shape generation, capable of producing high-fidelity shapes with colors that match the given text description. This work has several technical contributions. First, we decouple the shape and color predictions for learning features in both texts and shapes, and propose the word-level spatial transformer to correlate word features from text with spatial features from shape. Also, we design a cyclic loss to encourage consistency between text and shape, and introduce the shape IMLE to diversify the generated shapes. Further, we extend the framework to enable text-guided shape manipulation. Extensive experiments on the largest existing text-shape benchmark manifest the superiority of this work. The code and the models are available at https://github.com/liuzhengzhe/Towards-Implicit Text-Guided-Shape-Generation.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Builds on 68 of its 88 references that are in the map. First posted on arXiv 28 Mar 2022.

Builds on

68 of 88 references.

References · 68

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  7. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  8. 2016Improved Techniques for Training GANscited by 2,938
  9. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  10. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  11. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  12. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
38 more
  1. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  2. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  3. 2020Convolutional Occupancy Networkscited by 760
  4. 2016Generative Adversarial Text to Image Synthesiscited by 735
  5. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  6. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  7. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  8. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  9. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532
  10. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 460
  11. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
  12. 2020Local Implicit Grid Representations for 3D Scenescited by 372
  13. 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
  14. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 332
  15. 2020Neural Unsigned Distance Fields for Implicit Function Learningcited by 265
  16. 2020Learning Gradient Fields for Shape Generationcited by 245
  17. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  18. 2019BSP-Net: Generating Compact Meshes via Binary Space Partitioningcited by 205
  19. 2019Controllable Text-to-Image Generationcited by 187
  20. 2019MirrorGAN: Learning Text-to-image Generation by Redescriptioncited by 176
  21. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  22. 20193D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutionscited by 173
  23. 2018Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddingscited by 169
  24. 2019ManiGAN: Text-Guided Image Manipulationcited by 156
  25. 2020Deformed Implicit Field: Modeling 3D Shapes with Learned Dense Correspondencecited by 119
  26. 2019PQ-NET: A Generative Part Seq2Seq Network for 3D Shapescited by 118
  27. 2019SDM-NET: Deep Generative Network for Structured Deformable Meshcited by 113
  28. 2016Learning What and Where to Drawcited by 105
  29. 2018PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attentioncited by 105
  30. 2020SoftFlow: Probabilistic Framework for Normalizing Flow on Manifoldscited by 97
  31. 2018MeshNet: Mesh Neural Network for 3D Shape Representationcited by 95
  32. 2020Discrete Point Flow Networks for Efficient Point Cloud Generationcited by 87
  33. 2020Deep Implicit Templates for 3D Shape Representationcited by 86
  34. 2020DualSDF: Semantic Shape Manipulation using a Two-Level Representationcited by 82
  35. 2020PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representationscited by 81
  36. 2020DI-Fusion: Online Implicit 3D Reconstruction with Deep Priorscited by 70
  37. 2021Deep Implicit Moving Least-Squares Functions for 3D Reconstructioncited by 67
  38. 2018Diverse Image Synthesis from Semantic Layouts via Conditional IMLEcited by 60

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 14; 2023: 33; 2024: 15; 2025: 4; 2026: 2.

68 papers.

Sort:

By area:

  1. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  2. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  3. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  4. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  5. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 225
  6. 2022CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-trainingcited by 159
  7. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Modelscited by 151
  8. 2022Diffusion-SDF: Text-to-Shape via Voxelized Diffusioncited by 128
  9. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  10. 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 81
  11. 2022ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Modelcited by 62
  12. 2024Advances in 3D Generation: A Surveycited by 62
38 more
  1. 2022Understanding Pure CLIP Guidance for Voxel Grid NeRF Modelscited by 56
  2. 2022CLIP-Sculptor: Zero-Shot Generation of High-Fidelity and Diverse Shapes from Natural Languagecited by 46
  3. 2025CAD-Coder: An Open-Source Vision-Language Model for Computer-Aided Design Code Generationcited by 39
  4. 2023High-Fidelity 3D Face Generation from Natural Language Descriptionscited by 32
  5. 2023TAPS3D: Text-Guided 3D Textured Shape Generation from Pseudo Supervisioncited by 28
  6. 2022ISS: Image as Stepping Stone for Text-Guided 3D Shape Generationcited by 27
  7. 2023Neural Wavelet-domain Diffusion for 3D Shape Generation, Inversion, and Manipulationcited by 27
  8. 2024Text2LiDAR: Text-guided LiDAR Point Cloud Generation via Equirectangular Transformercited by 26
  9. 20233DQD: Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Processcited by 25
  10. 2023AI-Generated Content (AIGC) for Various Data Modalities: A Surveycited by 25
  11. 2024BrightDreamer: Generic 3D Gaussian Generative Framework for Fast Text-to-3D Synthesiscited by 20
  12. 2023EFEM: Equivariant Neural Field Expectation Maximization for 3D Object Segmentation Without Scene Supervisioncited by 18
  13. 2024InterFusion: Text-Driven Generation of 3D Human-Object Interactioncited by 18
  14. 2024DDMI: Domain-Agnostic Latent Diffusion Models for Synthesizing High-Quality Implicit Neural Representationscited by 16
  15. 2022PalGAN: Image Colorization with Palette Generative Adversarial Networkscited by 15
  16. 2023VPP: Efficient Conditional 3D Generation via Voxel-Point Progressive Representationcited by 14
  17. 2023T2TD: Text-3D Generation Model based on Prior Knowledge Guidancecited by 13
  18. 2024Text-to-3D Shape Generationcited by 13
  19. 2024NeuSDFusion: A Spatial-Aware Generative Model for 3D Shape Completion, Reconstruction, and Generationcited by 13
  20. 20233D VR Sketch Guided 3D Shape Prototyping and Explorationcited by 12
  21. 2023Towards High-Fidelity Text-Guided 3D Face Generation and Manipulation Using only Imagescited by 12
  22. 2023MinD-3D: Reconstruct High-quality 3D objects in Human Braincited by 11
  23. 2023CLIPXPlore: Coupled CLIP and Shape Spaces for 3D Shape Explorationcited by 9
  24. 20223D-TOGO: Towards Text-Guided Cross-Category 3D Object Generationcited by 8
  25. 2023Surf-D: Generating High-Quality Surfaces of Arbitrary Topologies Using Diffusion Modelscited by 8
  26. 2023Text-Guided 3D Face Synthesis -- From Generation to Editingcited by 8
  27. 2024CNS-Edit: 3D Shape Editing via Coupled Neural Shape Optimizationcited by 8
  28. 2024HyperSDFusion: Bridging Hierarchical Structures in Language and Geometry for Enhanced 3D Text2Shape Generationcited by 8
  29. 2023Spice-E : Structural Priors in 3D Diffusion using Cross-Entity Attentioncited by 7
  30. 2024Pushing Auto-regressive Models for 3D Shape Generation at Capacity and Scalabilitycited by 7
  31. 2022Language-Assisted 3D Feature Learning for Semantic Scene Understandingcited by 6
  32. 2022Neural Shape Compiler: A Unified Framework for Transforming between Text, Point Cloud, and Programcited by 6
  33. 2023TPA3D: Triplane Attention for Fast Text-to-3D Generationcited by 6
  34. 2024MinD-3D++: Advancing fMRI-Based 3D Reconstruction with High-Quality Textured Mesh Generation and a Comprehensive Datasetcited by 6
  35. 2023Architext: Language-Driven Generative Architecture Designcited by 5
  36. 2025TeRA: Rethinking Text-guided Realistic 3D Avatar Generationcited by 5
  37. 2023DreamStone: Image as Stepping Stone for Text-Guided 3D Shape Generationcited by 4
  38. 2023UniG3D: A Unified 3D Object Generation Datasetcited by 4

50 most cited of 68 All 68 (JSON)