mlsanityData to 7 Oct 2026

3DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors

arXiv 2403.02234 ↗First posted 4 Mar 2024cs.CVcited by 72

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present a two-stage text-to-3D generation system, namely 3DTopia, which generates high-quality general 3D assets within 5 minutes using hybrid diffusion priors. The first stage samples from a 3D diffusion prior directly learned from 3D data. Specifically, it is powered by a text-conditioned tri-plane latent diffusion model, which quickly generates coarse 3D samples for fast prototyping. The second stage utilizes 2D diffusion priors to further refine the texture of coarse 3D models from the first stage. The refinement consists of both latent and pixel space optimization for high-quality texture generation. To facilitate the training of the proposed system, we clean and caption the largest open-source 3D dataset, Objaverse, by combining the power of vision language models and large language models. Experiment results are reported qualitatively and quantitatively to show the performance of the proposed system. Our codes and models are available at https://github.com/3DTopia/3DTopia

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 56 of its 74 references that are in the map. First posted on arXiv 4 Mar 2024.

Builds on

56 of 74 references.

Closest ancestors · 20

Same direction, newest first

  1. 2023PrimDiffusion: Volumetric Primitives Diffusion for 3D Human Generation7 Dec
  2. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D28 Nov
  3. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3D4 Oct
  4. 2023Large-Vocabulary 3D Diffusion Model with Transformer14 Sep
  5. 2023IT3D: Improved Text-to-3D Generation with Explicit View Synthesis22 Aug
  6. 2023Points-to-3D: Bridging the Gap between Sparse Points and Shape-Controllable Text-to-3D Generation26 Jul
  7. 2023DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generation21 Jun
  8. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  9. 2023TextMesh: Generation of Realistic 3D Meshes From Text Prompts24 Apr
  10. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  11. 2023Vox-E: Text-guided Voxel Editing of 3D Objects21 Mar
  12. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
8 more
  1. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion12 Dec
  2. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusion2 Dec
  3. 20223D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Models1 Dec
  4. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  5. 20223D Neural Field Generation using Triplane Diffusion30 Nov
  6. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  7. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  8. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  7. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  8. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  9. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  10. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  11. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  12. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
24 more
  1. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  2. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  3. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  4. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  5. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  6. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  7. 2023Simple diffusion: End-to-end diffusion for high resolution imagescited by 413
  8. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  9. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  10. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  11. 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 262
  12. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  13. 2020Scan2Cap: Context-aware Dense Captioning in RGB-D Scanscited by 215
  14. 2023Text-To-4D Dynamic Scene Generationcited by 204
  15. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  16. 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 199
  17. 2023MeshDiffusion: Score-based Generative 3D Mesh Modelingcited by 198
  18. 2018Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddingscited by 169
  19. 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 151
  20. 2022EVA3D: Compositional 3D Human Generation from 2D Image Collectionscited by 111
  21. 2023Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinementcited by 106
  22. 2023FocalDreamer: Text-driven 3D Editing via Focal-fusion Assemblycited by 75
  23. 2023CityDreamer: Compositional Generative Model of Unbounded 3D Citiescited by 75
  24. 2023NeuManifold: Neural Watertight Manifold Reconstruction with Efficient and High-Quality Rendering Supportcited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 30; 2025: 33; 2026: 9.

72 papers.

Sort:

By area:

  1. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
  2. 2024MeshXL: Neural Coordinate Field for Generative 3D Foundation Modelscited by 100
  3. 20243DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusioncited by 100
  4. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 91
  5. 2024EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generationcited by 84
  6. 2024ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware Diffusion Guidancecited by 59
  7. 2025DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generationcited by 49
  8. 2025Efficient Part-level 3D Object Generation via Dual Volume Packingcited by 47
  9. 2025Ultra3D: Efficient and High-Fidelity 3D Generation with Part Attentioncited by 43
  10. 2025PhysX-3D: Physical-Grounded 3D Asset Generationcited by 40
  11. 2024SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAEcited by 38
  12. 2025PhysX-Anything: Simulation-Ready Physical 3D Assets from Single Imagecited by 32
38 more
  1. 2024Material Anything: Generating Materials for Any 3D Object via Diffusioncited by 31
  2. 2024ScaleDreamer: Scalable Text-to-3D Synthesis with Asynchronous Score Distillationcited by 27
  3. 2024AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animationcited by 25
  4. 2025Generating Physically Stable and Buildable Brick Structures from Textcited by 23
  5. 2024Chat2Layout: Interactive 3D Furniture Layout with a Multimodal LLMcited by 20
  6. 2024Phidias: A Generative Model for Creating 3D Content from Text, Image, and 3D Conditions with Reference-Augmented Diffusioncited by 19
  7. 20243D-Adapter: Geometry-Consistent Multi-View Diffusion for High-Quality 3D Generationcited by 19
  8. 2025Nautilus: Locality-aware Autoencoder for Scalable Mesh Generationcited by 18
  9. 2024Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with Image-Based Surface Representationcited by 17
  10. 2025SimpleFold: Folding Proteins is Simpler than You Thinkcited by 16
  11. 2024ThemeStation: Generating Theme-Aware 3D Assets from Few Exemplarscited by 14
  12. 2025Kiss3DGen: Repurposing Image Diffusion Models for 3D Asset Generationcited by 14
  13. 2025Hierarchical Neural Semantic Representation for 3D Semantic Correspondencecited by 14
  14. 2025WorldGrow: Generating Infinite 3D Worldcited by 14
  15. 2024A Survey on 3D Human Avatar Modeling -- From Reconstruction to Generationcited by 13
  16. 2024GaussianAnything: Interactive Point Cloud Flow Matching For 3D Object Generationcited by 12
  17. 20243DEnhancer: Consistent Multi-View Diffusion for 3D Enhancementcited by 12
  18. 20253DGen-Bench: Comprehensive Benchmark Suite for 3D Generative Modelscited by 12
  19. 2024GT23D-Bench: A Comprehensive General Text-to-3D Generation Benchmarkcited by 11
  20. 2024MUSES: 3D-Controllable Image Generation via Multi-Modal Agent Collaborationcited by 8
  21. 2024DetailGen3D: Generative 3D Geometry Enhancement via Data-Dependent Flowcited by 8
  22. 2025Zero-1-to-G: Taming Pretrained 2D Diffusion Model for Direct 3D Generationcited by 8
  23. 2025Towards Scalable Spatial Intelligence via 2D-to-3D Data Liftingcited by 8
  24. 2025AnchorFlow: Training-Free 3D Editing via Latent Anchor-Aligned Flowscited by 8
  25. 2024MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D Content Creationcited by 7
  26. 2024Benchmarking and Learning Multi-Dimensional Quality Evaluator for Text-to-3D Generationcited by 7
  27. 2025Disentangled Clothed Avatar Generation with Layered Representationcited by 7
  28. 2025MARS: Mesh AutoRegressive Model for 3D Shape Detailizationcited by 7
  29. 2025Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Datacited by 7
  30. 2025TRELLISWorld: Training-Free World Generation from Object Generatorscited by 7
  31. 2026PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objectscited by 7
  32. 2024GeneMAN: Generalizable Single-Image 3D Human Reconstruction from Multi-Source Human Datacited by 6
  33. 2025Advancing high-fidelity 3D and Texture Generation with 2.5D latentscited by 5
  34. 2024MicroDreamer: Efficient 3D Generation in ~20 Seconds by Score-based Iterative Reconstructioncited by 4
  35. 2025VaseVQA-3D: Benchmarking 3D VLMs on Ancient Greek Potterycited by 4
  36. 2024A3D: Does Diffusion Dream about 3D Alignment?cited by 3
  37. 2024ControLRM: Fast and Controllable 3D Generation via Large Reconstruction Modelcited by 3
  38. 2025DirectTriGS: Triplane-based Gaussian Splatting Field Representation for 3D Generationcited by 3

50 most cited of 72 All 72 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generation
  2. 2024DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D Data
  3. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion
  4. 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assets
  5. 2024Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior
  6. 2024GenesisTex2: Stable, Consistent and High-Quality Text-to-Texture Generation
  7. 2023UniDream: Unifying Diffusion Priors for Relightable Text-to-3D Generation
  8. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior