AToM: Amortized Text-to-Mesh using 2D Diffusion
Not on the map. Why?
Abstract (arXiv)
We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh framework optimized across multiple text prompts simultaneously. In contrast to existing text-to-3D methods that often entail time-consuming per-prompt optimization and commonly output representations other than polygonal meshes, AToM directly generates high-quality textured meshes in less than 1 second with around 10 times reduction in the training cost, and generalizes to unseen prompts. Our key idea is a novel triplane-based text-to-mesh architecture with a two-stage amortized optimization strategy that ensures stable training and enables scalability. Through extensive experiments on various prompt benchmarks, AToM significantly outperforms state-of-the-art amortized approaches with over 4 times higher accuracy (in DF415 dataset) and produces more distinguishable and higher-quality 3D outputs. AToM demonstrates strong generalizability, offering finegrained 3D assets for unseen interpolated prompts without further optimization during inference, unlike per-prompt solutions.
Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 49 of its 60 references that are in the map. First posted on arXiv 1 Feb 2024.
Builds on
49 of 60 references.
References · 49
- 2017Attention Is All You Needcited by 59,932
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
- 2022A ConvNet for the 2020scited by 3,545
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
- 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
- 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
37 more
- 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
- 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
- 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
- 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
- 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
- 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
- 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
- 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
- 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
- 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
- 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 449
- 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
- 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
- 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
- 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
- 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
- 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
- 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
- 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
- 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 181
- 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstructioncited by 180
- 20193D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutionscited by 173
- 2023Re-imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D, alleviate Janus problem and Beyondcited by 139
- 2023TextMesh: Generation of Realistic 3D Meshes From Text Promptscited by 135
- 20223DILG: Irregular Latent Grids for 3D Generative Modelingcited by 92
- 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
- 2022Tutorial on amortized optimizationcited by 83
- 2018Point Cloud GANcited by 79
- 2023InfiniCity: Infinite-Scale City Synthesiscited by 68
- 2017Improved Adversarial Systems for 3D Object Generation and Reconstructioncited by 66
- 2018Learning Descriptor Networks for 3D Shape Synthesis and Analysiscited by 56
- 2021Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketchescited by 54
- 2020Convolutional Generation of Textured 3D Meshescited by 52
- 2023SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priorscited by 43
- 2022Cross-Modal 3D Shape Generation and Manipulationcited by 27
- 2023HyperFields: Towards Zero-Shot Generation of NeRFs from Textcited by 13
Built on by
Citing papers per year: 2024: 13; 2025: 4; 2026: 1.
18 papers.
Sort:
By area:
- 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 214
- 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
- 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
- 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 88
- 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
- 2024LATTE3D: Large-scale Amortized Text-To-Enhanced3D Synthesiscited by 39
- 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 37
- 2026Lyra 2.0: Explorable Generative 3D Worldscited by 36
- 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
- 2024ScaleDreamer: Scalable Text-to-3D Synthesis with Asynchronous Score Distillationcited by 27
- 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 14
- 2024A Survey On Text-to-3D Contents Generation In The Wildcited by 13
6 more
- 2024Envision3D: One Image to 3D with Anchor Views Interpolationcited by 12
- 2024GT23D-Bench: A Comprehensive General Text-to-3D Generation Benchmarkcited by 11
- 2024GECO: Generative Image-to-3D within a SECOndcited by 6
- 2025DiMeR: Disentangled Mesh Reconstruction Modelcited by 6
- 2025ART-DECO: Arbitrary Text Guidance for 3D Detailizer Constructioncited by 6
- 2024SeMv-3D: Towards Concurrency of Semantic and Multi-view Consistency in General Text-to-3D Generationcited by 2
- 2026Lyra 2.0: Explorable Generative 3D Worlds14 Apr 2026
- 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation23 Sep 2025
- 2025ART-DECO: Arbitrary Text Guidance for 3D Detailizer Construction26 May 2025
- 2025DiMeR: Disentangled Mesh Reconstruction Model24 Apr 2025
- 2025Simulating the Real World: A Unified Survey of Multimodal Generative Models6 Mar 2025
- 2024GT23D-Bench: A Comprehensive General Text-to-3D Generation Benchmark13 Dec 2024
- 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers27 Nov 2024
- 2024SeMv-3D: Towards Concurrency of Semantic and Multi-view Consistency in General Text-to-3D Generation10 Oct 2024
- 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control17 Jul 2024
- 2024ScaleDreamer: Scalable Text-to-3D Synthesis with Asynchronous Score Distillation2 Jul 2024
- 2024GECO: Generative Image-to-3D within a SECOnd30 May 2024
- 2024LLMs Meet Multimodal Generation and Editing: A Survey29 May 2024
6 more
- 2024A Survey On Text-to-3D Contents Generation In The Wild15 May 2024
- 2024TC4D: Trajectory-Conditioned Text-to-4D Generation26 Mar 2024
- 2024LATTE3D: Large-scale Amortized Text-To-Enhanced3D Synthesis22 Mar 2024
- 2024Envision3D: One Image to 3D with Anchor Views Interpolation13 Mar 2024
- 2024V3D: Video Diffusion Models are Effective 3D Generators11 Mar 2024
- 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model8 Mar 2024
- 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
- 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
- 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 88
- 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 37
- 2024ScaleDreamer: Scalable Text-to-3D Synthesis with Asynchronous Score Distillationcited by 27
- 2024Envision3D: One Image to 3D with Anchor Views Interpolationcited by 12
- 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation23 Sep 2025
- 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers27 Nov 2024
- 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control17 Jul 2024
- 2024ScaleDreamer: Scalable Text-to-3D Synthesis with Asynchronous Score Distillation2 Jul 2024
- 2024Envision3D: One Image to 3D with Anchor Views Interpolation13 Mar 2024
- 2024V3D: Video Diffusion Models are Effective 3D Generators11 Mar 2024
- 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 214
- 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
- 2024LATTE3D: Large-scale Amortized Text-To-Enhanced3D Synthesiscited by 39
- 2026Lyra 2.0: Explorable Generative 3D Worldscited by 36
- 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
- 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 14
- 2024A Survey On Text-to-3D Contents Generation In The Wildcited by 13
- 2024GT23D-Bench: A Comprehensive General Text-to-3D Generation Benchmarkcited by 11
- 2024GECO: Generative Image-to-3D within a SECOndcited by 6
- 2025DiMeR: Disentangled Mesh Reconstruction Modelcited by 6
- 2025ART-DECO: Arbitrary Text Guidance for 3D Detailizer Constructioncited by 6
- 2024SeMv-3D: Towards Concurrency of Semantic and Multi-view Consistency in General Text-to-3D Generationcited by 2
- 2026Lyra 2.0: Explorable Generative 3D Worlds14 Apr 2026
- 2025ART-DECO: Arbitrary Text Guidance for 3D Detailizer Construction26 May 2025
- 2025DiMeR: Disentangled Mesh Reconstruction Model24 Apr 2025
- 2025Simulating the Real World: A Unified Survey of Multimodal Generative Models6 Mar 2025
- 2024GT23D-Bench: A Comprehensive General Text-to-3D Generation Benchmark13 Dec 2024
- 2024SeMv-3D: Towards Concurrency of Semantic and Multi-view Consistency in General Text-to-3D Generation10 Oct 2024
- 2024GECO: Generative Image-to-3D within a SECOnd30 May 2024
- 2024LLMs Meet Multimodal Generation and Editing: A Survey29 May 2024
- 2024A Survey On Text-to-3D Contents Generation In The Wild15 May 2024
- 2024TC4D: Trajectory-Conditioned Text-to-4D Generation26 Mar 2024
- 2024LATTE3D: Large-scale Amortized Text-To-Enhanced3D Synthesis22 Mar 2024
- 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model8 Mar 2024