mlsanityData to 7 Oct 2026

TAPS3D: Text-Guided 3D Textured Shape Generation from Pseudo Supervision

arXiv 2303.13273 ↗First posted 23 Mar 2023cs.CVcited by 28

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate an open research task of generating controllable 3D textured shapes from the given textual descriptions. Previous works either require ground truth caption labeling or extensive optimization time. To resolve these issues, we present a novel framework, TAPS3D, to train a text-guided 3D shape generator with pseudo captions. Specifically, based on rendered 2D images, we retrieve relevant words from the CLIP vocabulary and construct pseudo captions using templates. Our constructed captions provide high-level semantic supervision for generated 3D shapes. Further, in order to produce fine-grained textures and increase geometry diversity, we propose to adopt low-level image regularization to enable fake-rendered images to align with the real ones. During the inference phase, our proposed model can generate 3D textured shapes from the given text without any additional optimization. We conduct extensive experiments to analyze each of our proposed components and show the efficacy of our framework in generating high-fidelity 3D textured and text-relevant shapes.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 42 of its 56 references that are in the map. First posted on arXiv 23 Mar 2023.

Builds on

42 of 56 references.

References · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  9. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  10. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  11. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  12. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
30 more
  1. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  2. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  3. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  4. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  5. 2016Generative Adversarial Text to Image Synthesiscited by 735
  6. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  7. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  8. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  9. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  10. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
  11. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  12. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  13. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  14. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  15. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  16. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  17. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  18. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  19. 2019MirrorGAN: Learning Text-to-image Generation by Redescriptioncited by 176
  20. 20193D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutionscited by 173
  21. 2018Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddingscited by 169
  22. 2021CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directionscited by 78
  23. 2022Towards Implicit Text-Guided 3D Shape Generationcited by 68
  24. 2022ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Modelcited by 62
  25. 2022Understanding Pure CLIP Guidance for Voxel Grid NeRF Modelscited by 56
  26. 2021Learning Skeletal Articulations with Neural Blend Shapescited by 52
  27. 2021StyleMC: Multi-Channel Based Fast Text-Guided Image Generation and Manipulationcited by 40
  28. 20213D Pose Transfer with Correspondence Learning and Mesh Refinementcited by 30
  29. 2022Unsupervised 3D Pose Transfer with Cross Consistency and Dual Reconstructioncited by 21
  30. 2022Lafite2: Few-shot Text-to-Image Generationcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 13; 2024: 11; 2025: 3; 2026: 1.

28 papers.

Sort:

By area:

  1. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  2. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  3. 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 81
  4. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  5. 2024REACTO: Reconstructing Articulated Objects from a Single Videocited by 45
  6. 2024DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priorscited by 40
  7. 2023DreamControl: Control-Based Text-to-3D Generation with 3D Self-Priorcited by 39
  8. 2023VolumeDiffusion: Flexible Text-to-3D Generation with Efficient Volumetric Encodercited by 39
  9. 2024View Selection for 3D Captioning via Diffusion Rankingcited by 31
  10. 2025CADCrafter: Generating Computer-Aided Design Models from Unconstrained Imagescited by 31
  11. 2024Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Priorcited by 20
  12. 2023EXIM: A Hybrid Explicit-Implicit Representation for Text-Guided 3D Shape Generationcited by 16
16 more
  1. 2023MoDA: Modeling Deformable 3D Objects from Casual Videoscited by 13
  2. 2024Text-to-3D Shape Generationcited by 13
  3. 2023Single Mesh Diffusion Models with Field Latents for Texture Generationcited by 11
  4. 2023TextField3D: Towards Enhancing Open-Vocabulary 3D Generation with Noisy Text Fieldscited by 9
  5. 2023X-Dreamer: Creating High-quality 3D Content by Bridging the Domain Gap Between Text-to-2D and Text-to-3D Generationcited by 7
  6. 2023TPA3D: Triplane Attention for Fast Text-to-3D Generationcited by 6
  7. 2024Magic3DSketch: Create Colorful 3D Models From Sketch-Based 3D Modeling Guided by Text and Language-Image Pre-Trainingcited by 5
  8. 2024Deep Geometric Moments Promote Shape Consistency in Text-to-3D Generationcited by 5
  9. 2025TeRA: Rethinking Text-guided Realistic 3D Avatar Generationcited by 5
  10. 2024UV-free Texture Generation with Denoising and Geodesic Heat Diffusionscited by 3
  11. 2024GenUDC: High Quality 3D Mesh Generation with Unsigned Dual Contouring Representationcited by 3
  12. 2024Magic-Boost: Boost 3D Generation with Multi-View Conditioned Diffusioncited by 2
  13. 2023Text-Image Conditioned Diffusion for Consistent Text-to-3D Generationcited by 1
  14. 2024HOTS3D: Hyper-Spherical Optimal Transport for Semantic Alignment of Text-to-3D Generationcited by 1
  15. 2025DecompDreamer: A Composition-Aware Curriculum for Structured 3D Asset Generationcited by 1
  16. 2026RealCAD: Towards Real-World Image-to-CAD Reconstruction under Domain Shift and Parameter Biascited by 0