mlsanityData to 7 Oct 2026

AToM: Amortized Text-to-Mesh using 2D Diffusion

arXiv 2402.00867 ↗First posted 1 Feb 2024cs.CVcited by 18

Not on the map. Why?

Abstract (arXiv)

We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh framework optimized across multiple text prompts simultaneously. In contrast to existing text-to-3D methods that often entail time-consuming per-prompt optimization and commonly output representations other than polygonal meshes, AToM directly generates high-quality textured meshes in less than 1 second with around 10 times reduction in the training cost, and generalizes to unseen prompts. Our key idea is a novel triplane-based text-to-mesh architecture with a two-stage amortized optimization strategy that ensures stable training and enables scalability. Through extensive experiments on various prompt benchmarks, AToM significantly outperforms state-of-the-art amortized approaches with over 4 times higher accuracy (in DF415 dataset) and produces more distinguishable and higher-quality 3D outputs. AToM demonstrates strong generalizability, offering finegrained 3D assets for unseen interpolated prompts without further optimization during inference, unlike per-prompt solutions.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 49 of its 60 references that are in the map. First posted on arXiv 1 Feb 2024.

Builds on

49 of 60 references.

References · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  8. 2022A ConvNet for the 2020scited by 3,545
  9. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  10. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  11. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  12. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
37 more
  1. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  2. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  3. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  4. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  5. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  6. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  7. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  8. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  9. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  10. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  11. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  12. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 449
  13. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  14. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  15. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  16. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  17. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  18. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
  19. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  20. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  21. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 181
  22. 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstructioncited by 180
  23. 20193D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutionscited by 173
  24. 2023Re-imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D, alleviate Janus problem and Beyondcited by 139
  25. 2023TextMesh: Generation of Realistic 3D Meshes From Text Promptscited by 135
  26. 20223DILG: Irregular Latent Grids for 3D Generative Modelingcited by 92
  27. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  28. 2022Tutorial on amortized optimizationcited by 83
  29. 2018Point Cloud GANcited by 79
  30. 2023InfiniCity: Infinite-Scale City Synthesiscited by 68
  31. 2017Improved Adversarial Systems for 3D Object Generation and Reconstructioncited by 66
  32. 2018Learning Descriptor Networks for 3D Shape Synthesis and Analysiscited by 56
  33. 2021Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketchescited by 54
  34. 2020Convolutional Generation of Textured 3D Meshescited by 52
  35. 2023SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priorscited by 43
  36. 2022Cross-Modal 3D Shape Generation and Manipulationcited by 27
  37. 2023HyperFields: Towards Zero-Shot Generation of NeRFs from Textcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 4; 2026: 1.

18 papers.

Sort:

By area: