mlsanityData to 14 Oct 2026

Dual3D: Efficient and Consistent Text-to-3D Generation with Dual-mode Multi-view Latent Diffusion

arXiv 2405.09874 ↗First posted 16 May 2024cs.CVcited by 9

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present Dual3D, a novel text-to-3D generation framework that generates high-quality 3D assets from texts in only 1 minute.The key component is a dual-mode multi-view latent diffusion model. Given the noisy multi-view latents, the 2D mode can efficiently denoise them with a single latent denoising network, while the 3D mode can generate a tri-plane neural surface for consistent rendering-based denoising. Most modules for both modes are tuned from a pre-trained text-to-image latent diffusion model to circumvent the expensive cost of training from scratch. To overcome the high rendering cost during inference, we propose the dual-mode toggling inference strategy to use only 1/10 denoising steps with 3D mode, successfully generating a 3D asset in just 10 seconds without sacrificing quality. The texture of the 3D asset can be further enhanced by our efficient texture refinement process in a short time. Extensive experiments demonstrate that our method delivers state-of-the-art performance while significantly reducing generation time. Our project page is available at https://dual3d.github.io

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 58 of its 69 references that are in the map. First posted on arXiv 16 May 2024.

Builds on

58 of 69 references.

Closest ancestors · 15

Same direction, newest first

  1. 2023VolumeDiffusion: Flexible Text-to-3D Generation with Efficient Volumetric Encoder18 Dec
  2. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
  3. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  4. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  5. 2023Sparse3D: Distilling Multiview-Consistent Diffusion for Object Reconstruction from Sparse Views27 Aug
  6. 2023HD-Fusion: Detailed Text-to-3D Generation Leveraging Multiple Noise Estimation30 Jul
  7. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  8. 2023TextMesh: Generation of Realistic 3D Meshes From Text Prompts24 Apr
  9. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  10. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generation9 Mar
  11. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  12. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
3 more
  1. 20223D Neural Field Generation using Triplane Diffusion30 Nov
  2. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  3. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  9. 2022Scalable Diffusion Models with Transformerscited by 6,054
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  11. 2016Layer Normalizationcited by 5,669
  12. 2022Generative Adversarial Networkscited by 5,627
31 more
  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  2. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  3. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  4. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  5. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  6. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  7. 2018Group Normalizationcited by 1,524
  8. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  9. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  10. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  11. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  12. 2021Volume Rendering of Neural Implicit Surfacescited by 996
  13. 2020Neural Sparse Voxel Fieldscited by 959
  14. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  15. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  16. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
  17. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  18. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
  19. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  20. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  21. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 441
  22. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 332
  23. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  24. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 220
  25. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  26. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 214
  27. 2023Scalable 3D Captioning with Pretrained Modelscited by 203
  28. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 185
  29. 2023StreetSurf: Extending Multi-view Implicit Surface Reconstruction to Street Viewscited by 120
  30. 2023Noise-Free Score Distillationcited by 99
  31. 2022GRAM-HD: 3D-Consistent Image Generation at High Resolution with Generative Radiance Manifoldscited by 73

Built on by

Cited by year

Citing papers per year: 2024: 5; 2025: 4.

9 papers.

  1. 2024GOI: Find 3D Gaussians of Interest with an Optimizable Open-vocabulary Semantic-space Hyperplanecited by 53
  2. 2024Director3D: Real-world Camera Trajectory and 3D Scene Generation from Textcited by 50
  3. 2025Efficient Diffusion Models: A Surveycited by 35
  4. 2024SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesiscited by 34
  5. 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generationcited by 34
  6. 2025FlashWorld: High-quality 3D Scene Generation within Secondscited by 32
  7. 2025AR-1-to-3: Single Image to Consistent 3D Object Generation via Next-View Predictioncited by 15
  8. 2024MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D Content Creationcited by 7
  9. 2025Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Datacited by 7

Similar, not cited

Close in wording, no citation either way

  1. 2023HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance
  2. 2024ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models
  3. 2024Text-to-3D Generation by 2D Editing
  4. 2024GenesisTex2: Stable, Consistent and High-Quality Text-to-Texture Generation
  5. 2024BoostDream: Efficient Refining for High-Quality Text-to-3D Generation from Multi-View Diffusion
  6. 2023IT3D: Improved Text-to-3D Generation with Explicit View Synthesis
  7. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model
  8. 20233D-CLFusion: Fast Text-to-3D Rendering with Contrastive Latent Diffusion