mlsanityData to 7 Oct 2026

ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Model

arXiv 2207.09446 ↗First posted 19 Jul 2022cs.CV cs.AIcited by 62

Not on the map. Why?

Abstract (arXiv)

We present ShapeCrafter, a neural network for recursive text-conditioned 3D shape generation. Existing methods to generate text-conditioned 3D shapes consume an entire text prompt to generate a 3D shape in a single step. However, humans tend to describe shapes recursively-we may start with an initial description and progressively add details based on intermediate results. To capture this recursive process, we introduce a method to generate a 3D shape distribution, conditioned on an initial phrase, that gradually evolves as more phrases are added. Since existing datasets are insufficient for training this approach, we present Text2Shape++, a large dataset of 369K shape-text pairs that supports recursive shape generation. To capture local details that are often used to refine shape descriptions, we build on top of vector-quantized deep implicit functions that generate a distribution of high-quality shapes. Results show that our method can generate shapes consistent with text descriptions, and shapes evolve gradually as more phrases are added. Our method supports shape editing, extrapolation, and can enable new applications in human-machine collaboration for creative design.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Builds on 44 of its 61 references that are in the map. First posted on arXiv 19 Jul 2022.

Builds on

44 of 61 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  8. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  9. 2017Neural Discrete Representation Learningcited by 4,650
  10. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
32 more
  1. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  2. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  3. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  4. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  5. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  6. 2020Convolutional Occupancy Networkscited by 760
  7. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  8. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  9. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  10. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  11. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  12. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  13. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  14. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  15. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  16. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  17. 2018Constituency Parsing with a Self-Attentive Encodercited by 196
  18. 2019Controllable Text-to-Image Generationcited by 187
  19. 2018Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddingscited by 169
  20. 2014Deep Convolutional Neural Fields for Depth Estimation from a Single Imagecited by 106
  21. 2022ShapeFormer: Transformer-based Shape Completion via Sparse Representationcited by 106
  22. 2018Point Cloud GANcited by 79
  23. 2022Towards Implicit Text-Guided 3D Shape Generationcited by 68
  24. 2016Unsupervised Learning of 3D Structure from Imagescited by 67
  25. 2021Improving Text-to-Image Synthesis Using Contrastive Learningcited by 62
  26. 2019ShapeGlot: Learning Language for Shape Differentiationcited by 54
  27. 2019Voxel2Mesh: 3D Mesh Model Generation from Volumetric Datacited by 52
  28. 2021Adversarial Text-to-Image Synthesis: A Reviewcited by 52
  29. 2018Scan2Mesh: From Unstructured Range Scans to 3D Meshescited by 51
  30. 2020CNNs on Surfaces using Rotation-Equivariant Featurescited by 42
  31. 2021PixelTransformer: Sample Conditioned Signal Generationcited by 12
  32. 2019PointWise: An Unsupervised Point-wise Feature Learning Networkcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 35; 2024: 14; 2025: 7; 2026: 1.

62 papers.

Sort:

By area:

  1. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  2. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  3. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  4. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  5. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 181
  6. 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generationcited by 129
  7. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  8. 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 81
  9. 2023SKED: Sketch-guided Text-based 3D Editingcited by 72
  10. 2023SALAD: Part-Level Latent Diffusion for 3D Shape Generation and Manipulationcited by 66
  11. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  12. 2022CLIP-Sculptor: Zero-Shot Generation of High-Fidelity and Diverse Shapes from Natural Languagecited by 46
38 more
  1. 2023ShapeGPT: 3D Shape Generation with A Unified Multi-modal Language Modelcited by 38
  2. 2023SHAP-EDITOR: Instruction-guided Latent 3D Editing in Secondscited by 36
  3. 20233D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillationcited by 34
  4. 20223D Highlighter: Localizing Regions on 3D Shapes via Text Descriptionscited by 32
  5. 2024Generative Human Motion Stylization in Latent Spacecited by 30
  6. 2023TAPS3D: Text-Guided 3D Textured Shape Generation from Pseudo Supervisioncited by 28
  7. 2024A Comprehensive Survey on 3D Content Generationcited by 28
  8. 2023Multi-modal Machine Learning in Engineering Design: A Review and Future Directionscited by 27
  9. 20233DQD: Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Processcited by 25
  10. 2024ChatGarment: Garment Estimation, Generation and Editing via Large Language Modelscited by 22
  11. 2024ArtFormer: Controllable Generation of Diverse 3D Articulated Objectscited by 21
  12. 2024BrightDreamer: Generic 3D Gaussian Generative Framework for Fast Text-to-3D Synthesiscited by 20
  13. 2023EXIM: A Hybrid Explicit-Implicit Representation for Text-Guided 3D Shape Generationcited by 16
  14. 2023Sketch-A-Shape: Zero-Shot Sketch-to-3D Shape Generationcited by 14
  15. 2023VPP: Efficient Conditional 3D Generation via Voxel-Point Progressive Representationcited by 14
  16. 2024Text-to-3D Shape Generationcited by 13
  17. 20233D VR Sketch Guided 3D Shape Prototyping and Explorationcited by 12
  18. 2023Towards High-Fidelity Text-Guided 3D Face Generation and Manipulation Using only Imagescited by 12
  19. 2023Aladdin: Zero-Shot Hallucination of Stylized 3D Assets from Abstract Scene Descriptionscited by 11
  20. 2023CLIPXPlore: Coupled CLIP and Shape Spaces for 3D Shape Explorationcited by 9
  21. 2023TextField3D: Towards Enhancing Open-Vocabulary 3D Generation with Noisy Text Fieldscited by 9
  22. 2023Image Content Generation with Causal Reasoningcited by 9
  23. 2023Surf-D: Generating High-Quality Surfaces of Arbitrary Topologies Using Diffusion Modelscited by 8
  24. 2024CNS-Edit: 3D Shape Editing via Coupled Neural Shape Optimizationcited by 8
  25. 2024HyperSDFusion: Bridging Hierarchical Structures in Language and Geometry for Enhanced 3D Text2Shape Generationcited by 8
  26. 2023Image is All You Need to Empower Large-scale Diffusion Models for In-Domain Generationcited by 7
  27. 2024Benchmarking and Learning Multi-Dimensional Quality Evaluator for Text-to-3D Generationcited by 7
  28. 2022Neural Shape Compiler: A Unified Framework for Transforming between Text, Point Cloud, and Programcited by 6
  29. 2023TPA3D: Triplane Attention for Fast Text-to-3D Generationcited by 6
  30. 2024Don't Mesh with Me: Generating Constructive Solid Geometry Instead of Meshes by Fine-Tuning a Code-Generation LLMcited by 6
  31. 2023UniG3D: A Unified 3D Object Generation Datasetcited by 4
  32. 2025Recent Advances in 3D Object and Scene Generation: A Surveycited by 4
  33. 2023Text-guided Image-and-Shape Editing and Generation: A Short Surveycited by 3
  34. 2023Looking at words and points with attention: a benchmark for text-to-shape coherencecited by 3
  35. 2024Exploring the Potentials and Challenges of Deep Generative Models in Product Design Conceptioncited by 3
  36. 2025Walking the Schr"odinger Bridge: A Direct Trajectory for Text-to-3D Generationcited by 3
  37. 2024Toward Scene Graph and Layout Guided Complex 3D Scene Generationcited by 2
  38. 2025DM-OSVP++: One-Shot View Planning Using 3D Diffusion Models for Active RGB-Based Object Reconstructioncited by 2

50 most cited of 62 All 62 (JSON)