mlsanityData to 7 Oct 2026

GT23D-Bench: A Comprehensive General Text-to-3D Generation Benchmark

arXiv 2412.09997 ↗First posted 13 Dec 2024cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Text-to-3D (T23D) generation has emerged as a crucial visual generation task, aiming at synthesizing 3D content from textual descriptions. Studies of this task are currently shifting from per-scene T23D, which requires optimization of the model for every content generated, to General T23D (GT23D), which requires only one pre-trained model to generate different content without re-optimization, for more generalized and efficient 3D generation. Despite notable advancements, GT23D is severely bottlenecked by two interconnected challenges: the lack of high-quality, large-scale training data and the prevalence of evaluation metrics that overlook intrinsic 3D properties. Existing datasets often suffer from incomplete annotations, noisy organization, and inconsistent quality, while current evaluations rely heavily on 2D image-text similarity or scoring, failing to thoroughly assess 3D geometric integrity and semantic relevance. To address these fundamental gaps, we introduce GT23D-Bench, the first comprehensive benchmark specifically designed for GT23D training and evaluation. We first construct a high-quality dataset of 400K 3D assets, featuring diverse visual annotations (70M+ visual samples) and multi-granularity hierarchical captions (1M+ descriptions) to foster robust semantic learning. Second, we propose a comprehensive evaluation suite with 10 metrics assessing both text-3D alignment and 3D visual quality at multiple levels. Crucially, we demonstrate through rigorous experiments that our proposed metrics exhibit significantly higher correlation with human judgment compared to existing methods. Our in-depth analysis of eight leading GT23D models using this benchmark provides the community with critical insights into current model capabilities and their shared failure modes. GT23D-Bench will be publicly available to facilitate rigorous and reproducible research.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 59 of its 72 references that are in the map. First posted on arXiv 13 Dec 2024.

Builds on

59 of 72 references.

References · 59

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  9. 2016Improved Techniques for Training GANscited by 2,938
  10. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  11. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  12. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
38 more
  1. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  2. 2024Depth Anything V2cited by 1,385
  3. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  4. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  5. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  6. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  7. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  8. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  9. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  10. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  11. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  12. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  13. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  14. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
  15. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  16. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  17. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  18. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  19. 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 324
  20. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  21. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
  22. 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 276
  23. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
  24. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
  25. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 231
  26. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
  27. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  28. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 181
  29. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  30. 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
  31. 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 151
  32. 2021Simple multi-dataset detectioncited by 126
  33. 2023FETV: A Benchmark for Fine-Grained Evaluation of Open-Domain Text-to-Video Generationcited by 119
  34. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  35. 2023LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluationcited by 85
  36. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priorscited by 72
  37. 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
  38. 2016A Large Dataset of Object Scanscited by 64

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 5.

11 papers.