# Instant3D: Instant Text-to-3D Generation

arXiv: https://arxiv.org/abs/2311.08403 · HTML: https://mlsanity.com/paper/2311.08403/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 59 references that are in the map.
- First posted on arXiv 14 Nov 2023.

## Builds on (most cited first, 30 of 54)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [Point-E: A System for Generating 3D Point Clouds from Complex Prompts](https://mlsanity.com/paper/2212.08751/) (2022)

## Built on by (most cited first, 30 of 45)

- [LidarDM: Generative LiDAR Simulation in a Generated World](https://mlsanity.com/paper/2404.02903/) (2024)
- [REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignment](https://mlsanity.com/paper/2405.18525/) (2024)
- [LATTE3D: Large-scale Amortized Text-To-Enhanced3D Synthesis](https://mlsanity.com/paper/2403.15385/) (2024)
- [Dynamic 3D Gaussian Fields for Urban Areas](https://mlsanity.com/paper/2406.03175/) (2024)
- [GoodDrag: Towards Good Practices for Drag Editing with Diffusion Models](https://mlsanity.com/paper/2404.07206/) (2024)
- [EventGPT: Event Stream Understanding with Multimodal Large Language Models](https://mlsanity.com/paper/2412.00832/) (2024)
- [ScaleDreamer: Scalable Text-to-3D Synthesis with Asynchronous Score Distillation](https://mlsanity.com/paper/2407.02040/) (2024)
- [CrossViewDiff: A Cross-View Diffusion Model for Satellite-to-Street View Synthesis](https://mlsanity.com/paper/2408.14765/) (2024)
- [3DSceneEditor: Controllable 3D Scene Editing with Gaussian Splatting](https://mlsanity.com/paper/2412.01583/) (2024)
- [Synthetic-to-Real Self-supervised Robust Depth Estimation via Learning with Motion and Structure Priors](https://mlsanity.com/paper/2503.20211/) (2025)
- [BrightDreamer: Generic 3D Gaussian Generative Framework for Fast Text-to-3D Synthesis](https://mlsanity.com/paper/2403.11273/) (2024)
- [Gaussian Splatting Decoder for 3D-aware Generative Adversarial Networks](https://mlsanity.com/paper/2404.10625/) (2024)
- [A Survey On Text-to-3D Contents Generation In The Wild](https://mlsanity.com/paper/2405.09431/) (2024)
- [Text-to-3D Shape Generation](https://mlsanity.com/paper/2403.13289/) (2024)
- [AWOL: Analysis WithOut synthesis using Language](https://mlsanity.com/paper/2404.03042/) (2024)
- [Sync4D: Video Guided Controllable Dynamics for Physics-Based 4D Generation](https://mlsanity.com/paper/2405.16849/) (2024)
- [JointDreamer: Ensuring Geometry Consistency and Text Congruence in Text-to-3D Generation via Joint Score Distillation](https://mlsanity.com/paper/2407.12291/) (2024)
- [Safe-Sora: Safe Text-to-Video Generation via Graphical Watermarking](https://mlsanity.com/paper/2505.12667/) (2025)
- [TriNeRFLet: A Wavelet Based Triplane NeRF Representation](https://mlsanity.com/paper/2401.06191/) (2024)
- [Benchmarking and Learning Multi-Dimensional Quality Evaluator for Text-to-3D Generation](https://mlsanity.com/paper/2412.11170/) (2024)
- [Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Data](https://mlsanity.com/paper/2503.21694/) (2025)
- [DiMeR: Disentangled Mesh Reconstruction Model](https://mlsanity.com/paper/2504.17670/) (2025)
- [Apply Hierarchical-Chain-of-Generation to Complex Attributes Text-to-3D Generation](https://mlsanity.com/paper/2505.05505/) (2025)
- [Distilling Multi-view Diffusion Models into 3D Generators](https://mlsanity.com/paper/2504.00457/) (2025)
- [ControLRM: Fast and Controllable 3D Generation via Large Reconstruction Model](https://mlsanity.com/paper/2410.09592/) (2024)
- [DirectTriGS: Triplane-based Gaussian Splatting Field Representation for 3D Generation](https://mlsanity.com/paper/2503.06900/) (2025)
- [Control and Realism: Best of Both Worlds in Layout-to-Image without Training](https://mlsanity.com/paper/2506.15563/) (2025)
- [Towards Fine-Grained Text-to-3D Quality Assessment: A Benchmark and A Two-Stage Rank-Learning Metric](https://mlsanity.com/paper/2509.23841/) (2025)
- [CogniVerse: Revolutionizing Multi-Modal Retrieval-Augmented Generation with Cognitive Reflection and Geometric Reasoning](https://mlsanity.com/paper/2605.29602/) (2026)
- [EarthGen: Generating the World from Top-Down Views](https://mlsanity.com/paper/2409.01491/) (2024)

## Abstract (arXiv)

Text-to-3D generation has attracted much attention from the computer vision community. Existing methods mainly optimize a neural field from scratch for each text prompt, relying on heavy and repetitive training cost which impedes their practical deployment. In this paper, we propose a novel framework for fast text-to-3D generation, dubbed Instant3D. Once trained, Instant3D is able to create a 3D object for an unseen text prompt in less than one second with a single run of a feedforward network. We achieve this remarkable speed by devising a new network that directly constructs a 3D triplane from a text prompt. The core innovation of our Instant3D lies in our exploration of strategies to effectively inject text conditions into the network. In particular, we propose to combine three key mechanisms: cross-attention, style injection, and token-to-plane transformation, which collectively ensure precise alignment of the output with the input text. Furthermore, we propose a simple yet effective activation function, the scaled-sigmoid, to replace the original sigmoid function, which speeds up the training convergence by more than ten times. Finally, to address the Janus (multi-head) problem in 3D generation, we propose an adaptive Perp-Neg algorithm that can dynamically adjust its concept negation scales according to the severity of the Janus problem during training, effectively reducing the multi-head effect. Extensive experiments on a wide variety of benchmark datasets demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods both qualitatively and quantitatively, while achieving significantly better efficiency. The code, data, and models are available at https://github.com/ming1993li/Instant3DCodes.