mlsanityData to 14 Oct 2026

UniG3D: A Unified 3D Object Generation Dataset

arXiv 2306.10730 ↗First posted 19 Jun 2023cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

The field of generative AI has a transformative impact on various areas, including virtual reality, autonomous driving, the metaverse, gaming, and robotics. Among these applications, 3D object generation techniques are of utmost importance. This technique has unlocked fresh avenues in the realm of creating, customizing, and exploring 3D objects. However, the quality and diversity of existing 3D object generation methods are constrained by the inadequacies of existing 3D object datasets, including issues related to text quality, the incompleteness of multi-modal data representation encompassing 2D rendered images and 3D assets, as well as the size of the dataset. In order to resolve these issues, we present UniG3D, a unified 3D object generation dataset constructed by employing a universal data transformation pipeline on Objaverse and ShapeNet datasets. This pipeline converts each raw 3D model into comprehensive multi-modal data representation <text, image, point cloud, mesh> by employing rendering engines and multi-modal models. These modules ensure the richness of textual information and the comprehensiveness of data representation. Remarkably, the universality of our pipeline refers to its ability to be applied to any 3D dataset, as it only requires raw 3D data. The selection of data sources for our dataset is based on their scale and quality. Subsequently, we assess the effectiveness of our dataset by employing Point-E and SDFusion, two widely recognized methods for object generation, tailored to the prevalent 3D representations of point clouds and signed distance functions. Our dataset is available at: https://unig3d.github.io.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 63 of its 76 references that are in the map. First posted on arXiv 19 Jun 2023.

Builds on

63 of 76 references.

References · 63

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 2017Neural Discrete Representation Learningcited by 4,712
  9. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  11. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
38 more
  1. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  2. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  4. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  5. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  6. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  7. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  8. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  9. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  10. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 735
  11. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  12. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 699
  13. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  14. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  15. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  16. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 612
  17. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
  18. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532
  19. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  20. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  21. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 490
  22. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 488
  23. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  24. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  25. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  26. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 441
  27. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 385
  28. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 366
  29. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  30. 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 326
  31. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 311
  32. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  33. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 288
  34. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  35. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  36. 2020Learning Gradient Fields for Shape Generationcited by 245
  37. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  38. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 220

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2.

4 papers.

  1. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
  2. 2024Controllable Text-to-3D Generation via Surface-Aligned Gaussian Splattingcited by 26
  3. 2024Kestrel: 3D Multimodal LLM for Part-Aware Grounded Descriptioncited by 9
  4. 2023MVControl: Adding Conditional Control to Multi-view Diffusion for Controllable Text-to-3D Generationcited by 8