mlsanityData to 14 Oct 2026

LATTE3D: Large-scale Amortized Text-To-Enhanced3D Synthesis

arXiv 2403.15385 ↗First posted 22 Mar 2024cs.CV cs.AI cs.GRcited by 39

Not on the map. Why?

Abstract (arXiv)

Recent text-to-3D generation approaches produce impressive 3D results but require time-consuming optimization that can take up to an hour per prompt. Amortized methods like ATT3D optimize multiple prompts simultaneously to improve efficiency, enabling fast text-to-3D synthesis. However, they cannot capture high-frequency geometry and texture details and struggle to scale to large prompt sets, so they generalize poorly. We introduce LATTE3D, addressing these limitations to achieve fast, high-quality generation on a significantly larger prompt set. Key to our method is 1) building a scalable architecture and 2) leveraging 3D data during optimization through 3D-aware diffusion priors, shape regularization, and model initialization to achieve robustness to diverse and complex training prompts. LATTE3D amortizes both neural field and textured surface generation to produce highly detailed textured meshes in a single forward pass. LATTE3D generates 3D objects in 400ms, and can be further enhanced with fast test-time optimization.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Builds on 51 of its 64 references that are in the map. First posted on arXiv 22 Mar 2024.

Builds on

51 of 64 references.

References · 51

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  6. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2013Auto-Encoding Variational Bayescited by 4,053
  9. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  10. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  11. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  12. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
38 more
  1. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  2. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  3. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  4. 2021Volume Rendering of Neural Implicit Surfacescited by 996
  5. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
  6. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
  7. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  8. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 699
  9. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  10. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  11. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 560
  12. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  13. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
  14. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 521
  15. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  16. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  17. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 488
  18. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 460
  19. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  20. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 441
  21. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 385
  22. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 378
  23. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 366
  24. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  25. 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 269
  26. 2023Scalable 3D Captioning with Pretrained Modelscited by 203
  27. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  28. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 183
  29. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  30. 2023Flexible Isosurface Extraction for Gradient-Based Mesh Optimizationcited by 139
  31. 2018Inference Suboptimality in Variational Autoencoderscited by 132
  32. 2019Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functionscited by 105
  33. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  34. 2018Stochastic Hyperparameter Optimization through Hypernetworkscited by 73
  35. 2023Instant3D: Instant Text-to-3D Generationcited by 45
  36. 2020Inverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Datacited by 30
  37. 2019Meta-Amortized Variational Inference and Learningcited by 22
  38. 2024AToM: Amortized Text-to-Mesh using 2D Diffusioncited by 18

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 25; 2025: 11; 2026: 3.

39 papers.

Sort:

By area:

  1. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  2. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 154
  3. 2024LLaMA-Mesh: Unifying 3D Mesh Generation with Language Modelscited by 91
  4. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 77
  5. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  6. 2024InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Modelscited by 56
  7. 2024MeshFormer: High-Quality Mesh Generation with 3D-Guided Reconstruction Modelcited by 55
  8. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 44
  9. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 41
  10. 2024SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAEcited by 39
  11. 2024ScaleDreamer: Scalable Text-to-3D Synthesis with Asynchronous Score Distillationcited by 27
  12. 2024Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with Image-Based Surface Representationcited by 17
27 more
  1. 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 15
  2. 2026Addressable Memory for Video World Modelscited by 15
  3. 2024A Survey On Text-to-3D Contents Generation In The Wildcited by 14
  4. 2025Kiss3DGen: Repurposing Image Diffusion Models for 3D Asset Generationcited by 14
  5. 2024AWOL: Analysis WithOut synthesis using Languagecited by 12
  6. 2024Edify 3D: Scalable High-Quality 3D Asset Generationcited by 10
  7. 2024FlowDreamer: Exploring High Fidelity Text-to-3D Generation via Rectified Flowcited by 7
  8. 2024ARM: Appearance Reconstruction Model for Relightable 3D Generationcited by 7
  9. 2024MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D Content Creationcited by 7
  10. 2025Score Distillation Sampling for Audio: Source Separation, Synthesis, and Beyondcited by 6
  11. 2025ART-DECO: Arbitrary Text Guidance for 3D Detailizer Constructioncited by 6
  12. 2024Improving Hyperparameter Optimization with Checkpointed Model Weightscited by 5
  13. 2025LSD-3D: Large-Scale 3D Driving Scene Generation with Geometry Groundingcited by 5
  14. 2024Dream-in-Style: Text-to-3D Generation Using Stylized Score Distillationcited by 4
  15. 2024Speech to Reality: On-Demand Production using Natural Language, 3D Generative AI, and Discrete Robotic Assemblycited by 4
  16. 2025Distilling Multi-view Diffusion Models into 3D Generatorscited by 4
  17. 2024Meta 3D Gencited by 3
  18. 2024ControLRM: Fast and Controllable 3D Generation via Large Reconstruction Modelcited by 3
  19. 2025Cyc3D: Fine-grained Controllable 3D Generation via Cycle Consistency Regularizationcited by 3
  20. 2024Predicting the Encoding Error of SIRENscited by 2
  21. 2024ModeDreamer: Mode Guiding Score Distillation for Text-to-3D Generation using Reference Image Promptscited by 2
  22. 2025Text-to-3D Generation using Jensen-Shannon Score Distillationcited by 2
  23. 2025Track, Inpaint, Resplat: Subject-driven 3D and 4D Generation with Progressive Texture Infillingcited by 2
  24. 2025Text to Robotic Assembly of Multi Component Objects using 3D Generative AI and Vision Language Modelscited by 2
  25. 2024Ask, Pose, Unite: Scaling Data Acquisition for Close Interactions with Vision Language Modelscited by 1
  26. 2024ExCellGen: Fast, Controllable, Photorealistic 3D Scene Generation from a Single Real-World Exemplarcited by 0
  27. 2026VideoMDM: Towards 3D Human Motion Generation From 2D Supervisioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 20253D-LATTE: Latent Space 3D Editing from Textual Instructions
  2. 2024HexaGen3D: StableDiffusion is just one step away from Fast and Diverse Text-to-3D Generation
  3. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors
  4. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models
  5. 2024Dual3D: Efficient and Consistent Text-to-3D Generation with Dual-mode Multi-view Latent Diffusion
  6. 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assets
  7. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior
  8. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors