Meta 3D Gen
Not on the map. Why?
Abstract (arXiv)
We introduce Meta 3D Gen (3DGen), a new state-of-the-art, fast pipeline for text-to-3D asset generation. 3DGen offers 3D asset creation with high prompt fidelity and high-quality 3D shapes and textures in under a minute. It supports physically-based rendering (PBR), necessary for 3D asset relighting in real-world applications. Additionally, 3DGen supports generative retexturing of previously generated (or artist-created) 3D shapes using additional textual inputs provided by the user. 3DGen integrates key technical components, Meta 3D AssetGen and Meta 3D TextureGen, that we developed for text-to-3D and text-to-texture generation, respectively. By combining their strengths, 3DGen represents 3D objects simultaneously in three ways: in view space, in volumetric space, and in UV (or texture) space. The integration of these two techniques achieves a win rate of 68% with respect to the single-stage model. We compare 3DGen to numerous industry baselines, and show that it outperforms them in terms of prompt fidelity and visual quality for complex textual prompts, while being significantly faster.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 112 of its 126 references that are in the map. First posted on arXiv 2 Jul 2024.
Builds on
112 of 126 references.
References · 112
- 2017Attention Is All You Needcited by 59,932
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2021Zero-Shot Text-to-Image Generationcited by 4,276
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
38 more
- 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
- 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
- 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
- 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
- 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
- 2021Volume Rendering of Neural Implicit Surfacescited by 990
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
- 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
- 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
- 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
- 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
- 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
- 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
- 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
- 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
- 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
- 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
- 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 611
- 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
- 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
- 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
- 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 516
- 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
- 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
- 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 446
- 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 425
- 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
- 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
- 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
- 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
- 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
- 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
- 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 344
- 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
- 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 332
50 most cited of 112 All 112 (JSON)
Built on by
Citing papers per year: 2024: 2; 2025: 0; 2026: 1.
3 papers.