mlsanityData to 7 Oct 2026

TPA3D: Triplane Attention for Fast Text-to-3D Generation

arXiv 2312.02647 ↗First posted 5 Dec 2023cs.CVcited by 6

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Due to the lack of large-scale text-3D correspondence data, recent text-to-3D generation works mainly rely on utilizing 2D diffusion models for synthesizing 3D data. Since diffusion-based methods typically require significant optimization time for both training and inference, the use of GAN-based models would still be desirable for fast 3D generation. In this work, we propose Triplane Attention for text-guided 3D generation (TPA3D), an end-to-end trainable GAN-based deep learning model for fast text-to-3D generation. With only 3D shape data and their rendered 2D images observed during training, our TPA3D is designed to retrieve detailed visual descriptions for synthesizing the corresponding 3D mesh data. This is achieved by the proposed attention mechanisms on the extracted sentence and word-level text features. In our experiments, we show that TPA3D generates high-quality 3D textured shapes aligned with fine-grained descriptions, while impressive computation efficiency can be observed.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 52 of its 62 references that are in the map. First posted on arXiv 5 Dec 2023.

Builds on

52 of 62 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Large-Vocabulary 3D Diffusion Model with Transformer14 Sep
  2. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models28 Dec
  3. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  4. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  5. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  9. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  10. 2017Neural Discrete Representation Learningcited by 4,650
  11. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,276
35 more
  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  2. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  3. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  7. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  8. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  9. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  10. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  11. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  12. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  13. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  14. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  15. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  16. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  17. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  18. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  19. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  20. 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 324
  21. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
  22. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  23. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  24. 2018Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddingscited by 169
  25. 2022Diffusion-SDF: Text-to-Shape via Voxelized Diffusioncited by 128
  26. 2023GALIP: Generative Adversarial CLIPs for Text-to-Image Synthesiscited by 84
  27. 2022Towards Implicit Text-Guided 3D Shape Generationcited by 68
  28. 2022ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Modelcited by 62
  29. 2023Gen2Sim: Scaling up Robot Learning in Simulation with Generative Modelscited by 62
  30. 2018Implicit Maximum Likelihood Estimationcited by 52
  31. 2022CLIP-Sculptor: Zero-Shot Generation of High-Fidelity and Diverse Shapes from Natural Languagecited by 46
  32. 2021Learning Generative Models of Textured 3D Meshes from Real-World Imagescited by 44
  33. 2023TAPS3D: Text-Guided 3D Textured Shape Generation from Pseudo Supervisioncited by 28
  34. 2020Fit2Form: 3D Generative Model for Robot Gripper Form Designcited by 21
  35. 2023TextField3D: Towards Enhancing Open-Vocabulary 3D Generation with Noisy Text Fieldscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 1.

6 papers.

  1. 20243DSceneEditor: Controllable 3D Scene Editing with Gaussian Splattingcited by 20
  2. 2025Towards Scalable and Consistent 3D Editingcited by 9
  3. 2025Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Datacited by 7
  4. 2025HyPlaneHead: Rethinking Tri-plane-like Representations in Full-Head Image Synthesiscited by 6
  5. 2025Category-Aware 3D Object Composition with Disentangled Texture and Shape Multi-view Diffusioncited by 2
  6. 2026HandDreamer: Zero-Shot Text to 3D Hand Model Generation using Corrective Hand Shape Guidancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023IT3D: Improved Text-to-3D Generation with Explicit View Synthesis
  2. 2023PI3D: Efficient Text-to-3D Generation with Pseudo-Image Diffusion
  3. 2023DiffusionGAN3D: Boosting Text-guided 3D Generation and Domain Adaptation by Combining 3D GANs and Diffusion Priors
  4. 2024Portrait3D: Text-Guided High-Quality 3D Portrait Generation Using Pyramid Representation and GANs Prior
  5. 2023ET3D: Efficient Text-to-3D Generation via Multi-View Distillation
  6. 2024GANFusion: Feed-Forward Text-to-3D with Diffusion in GAN Space
  7. 2023Efficient Text-Guided 3D-Aware Portrait Generation with Score Distillation Sampling on Distribution
  8. 2023Sherpa3D: Boosting High-Fidelity Text-to-3D Generation via Coarse 3D Prior