GT23D-Bench: A Comprehensive General Text-to-3D Generation Benchmark
Not on the map. Why?
Abstract (arXiv)
Text-to-3D (T23D) generation has emerged as a crucial visual generation task, aiming at synthesizing 3D content from textual descriptions. Studies of this task are currently shifting from per-scene T23D, which requires optimization of the model for every content generated, to General T23D (GT23D), which requires only one pre-trained model to generate different content without re-optimization, for more generalized and efficient 3D generation. Despite notable advancements, GT23D is severely bottlenecked by two interconnected challenges: the lack of high-quality, large-scale training data and the prevalence of evaluation metrics that overlook intrinsic 3D properties. Existing datasets often suffer from incomplete annotations, noisy organization, and inconsistent quality, while current evaluations rely heavily on 2D image-text similarity or scoring, failing to thoroughly assess 3D geometric integrity and semantic relevance. To address these fundamental gaps, we introduce GT23D-Bench, the first comprehensive benchmark specifically designed for GT23D training and evaluation. We first construct a high-quality dataset of 400K 3D assets, featuring diverse visual annotations (70M+ visual samples) and multi-granularity hierarchical captions (1M+ descriptions) to foster robust semantic learning. Second, we propose a comprehensive evaluation suite with 10 metrics assessing both text-3D alignment and 3D visual quality at multiple levels. Crucially, we demonstrate through rigorous experiments that our proposed metrics exhibit significantly higher correlation with human judgment compared to existing methods. Our in-depth analysis of eight leading GT23D models using this benchmark provides the community with critical insights into current model capabilities and their shared failure modes. GT23D-Bench will be publicly available to facilitate rigorous and reproducible research.
Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 59 of its 72 references that are in the map. First posted on arXiv 13 Dec 2024.
Builds on
59 of 72 references.
References · 59
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023GPT-4 Technical Reportcited by 18,132
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Visual Instruction Tuningcited by 7,792
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
- 2016Improved Techniques for Training GANscited by 2,938
- 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
38 more
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
- 2024Depth Anything V2cited by 1,385
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
- 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
- 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
- 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
- 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
- 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
- 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
- 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
- 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
- 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
- 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
- 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
- 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
- 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 324
- 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
- 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
- 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 276
- 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
- 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
- 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 231
- 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
- 2023Scalable 3D Captioning with Pretrained Modelscited by 201
- 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 181
- 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
- 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
- 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 151
- 2021Simple multi-dataset detectioncited by 126
- 2023FETV: A Benchmark for Fine-Grained Evaluation of Open-Domain Text-to-Video Generationcited by 119
- 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
- 2023LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluationcited by 85
- 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priorscited by 72
- 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
- 2016A Large Dataset of Object Scanscited by 64
50 most cited of 59 All 59 (JSON)
Built on by
Citing papers per year: 2025: 6; 2026: 5.
11 papers.
- 2025TexVerse: A Universe of 3D Objects with High-Resolution Texturescited by 41
- 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 14
- 20254DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Modelscited by 10
- 2026WorldCoder-Bench: Benchmarking Physically Grounded 3D World Synthesiscited by 6
- 2025Hi3DEval: Advancing 3D Generation Evaluation with Hierarchical Validitycited by 3
- 2025Large Material Gaussian Model for Relightable 3D Generationcited by 2
- 2025Towards Fine-Grained Text-to-3D Quality Assessment: A Benchmark and A Two-Stage Rank-Learning Metriccited by 2
- 2026DB-3DME: From Dataset to Benchmark for Human-aligned Automatic 3D Mesh Evaluationcited by 2
- 20263D-DefectBench: A Controlled Factorial Study of Vision-Language Model Evaluation Pipelines for Fine-Grained 3D Generation Defectscited by 1
- 2026Can 4D Foundation Models Remember?cited by 0
- 2026WorldBench: Evaluating LLMs on Three.js Voxel World Generationcited by 0
- 2026WorldBench: Evaluating LLMs on Three.js Voxel World Generation7 Oct 2026
- 2026Can 4D Foundation Models Remember?17 Sep 2026
- 20263D-DefectBench: A Controlled Factorial Study of Vision-Language Model Evaluation Pipelines for Fine-Grained 3D Generation Defects12 Jul 2026
- 2026DB-3DME: From Dataset to Benchmark for Human-aligned Automatic 3D Mesh Evaluation8 Jun 2026
- 2026WorldCoder-Bench: Benchmarking Physically Grounded 3D World Synthesis1 Jun 2026
- 20254DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Models25 Nov 2025
- 2025Towards Fine-Grained Text-to-3D Quality Assessment: A Benchmark and A Two-Stage Rank-Learning Metric28 Sep 2025
- 2025Large Material Gaussian Model for Relightable 3D Generation26 Sep 2025
- 2025TexVerse: A Universe of 3D Objects with High-Resolution Textures14 Aug 2025
- 2025Hi3DEval: Advancing 3D Generation Evaluation with Hierarchical Validity7 Aug 2025
- 2025Simulating the Real World: A Unified Survey of Multimodal Generative Models6 Mar 2025