mlsanityData to 7 Oct 2026

Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with Image-Based Surface Representation

arXiv 2409.03718 ↗First posted 5 Sep 2024cs.CV cs.GRcited by 17

Not on the map. Why?

Abstract (arXiv)

Generating high-quality 3D objects from textual descriptions remains a challenging problem due to computational cost, the scarcity of 3D data, and complex 3D representations. We introduce Geometry Image Diffusion (GIMDiffusion), a novel Text-to-3D model that utilizes geometry images to efficiently represent 3D shapes using 2D images, thereby avoiding the need for complex 3D-aware architectures. By integrating a Collaborative Control mechanism, we exploit the rich 2D priors of existing Text-to-Image models such as Stable Diffusion. This enables strong generalization even with limited 3D training data (allowing us to use only high-quality training data) as well as retaining compatibility with guidance techniques such as IPAdapter. In short, GIMDiffusion enables the generation of 3D assets at speeds comparable to current Text-to-Image models. The generated objects consist of semantically meaningful, separate parts and include internal structures, enhancing both usability and versatility.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 66 of its 83 references that are in the map. First posted on arXiv 5 Sep 2024.

Builds on

66 of 83 references.

References · 66

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  9. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  10. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  11. 2022Flow Matching for Generative Modelingcited by 5,426
  12. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
38 more
  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  3. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  5. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  6. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  7. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  8. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  9. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  10. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  11. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  12. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  13. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  14. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  15. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  16. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
  17. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  18. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  19. 2021Neural Fields in Visual Computing and Beyondcited by 539
  20. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 516
  21. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  22. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimationcited by 323
  23. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 308
  24. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  25. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 254
  26. 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 227
  27. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 214
  28. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  29. 2023Flexible Isosurface Extraction for Gradient-Based Mesh Optimizationcited by 138
  30. 2023HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidancecited by 109
  31. 2023Noise-Free Score Distillationcited by 99
  32. 2022SDF-StyleGAN: Implicit SDF-Based StyleGAN for 3D Shape Generationcited by 92
  33. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  34. 2023DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimationcited by 81
  35. 2021Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifoldscited by 72
  36. 2023Free3D: Consistent Novel View Synthesis without 3D Representationcited by 72
  37. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priorscited by 72
  38. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 6.

17 papers.

Sort:

By area:

  1. 2025RomanTex: Decoupling 3D-aware Rotary Positional Embedded Multi-Attention Network for Texture Synthesiscited by 20
  2. 2025Zero-1-to-G: Taming Pretrained 2D Diffusion Model for Direct 3D Generationcited by 8
  3. 2025UVGS: Reimagining Unstructured 3D Gaussian Splatting using UV Mappingcited by 8
  4. 2025Repurposing 2D Diffusion Models with Gaussian Atlas for 3D Generationcited by 7
  5. 2025Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Datacited by 7
  6. 2025SPGen: Spherical Projection as Consistent and Flexible Representation for Single Image 3D Shape Generationcited by 5
  7. 2025Generative Human Geometry Distributioncited by 4
  8. 2025GarmageNet: A Multimodal Generative Framework for Sewing Pattern Design and Generic Garment Modelingcited by 4
  9. 2026MeshTailor: Cutting Seams via Generative Mesh Traversalcited by 4
  10. 2025SuperCarver: Texture-Consistent 3D Geometry Super-Resolution for High-Fidelity Surface Detail Generationcited by 2
  11. 2025GenVDM: Generating Vector Displacement Maps From a Single Imagecited by 1
  12. 2025Repurposing 2D Diffusion Models for 3D Shape Completioncited by 1
5 more
  1. 2026SwiftTailor: Efficient 3D Garment Generation with Geometry Image Representationcited by 0
  2. 2026CompoSE: Compositional Synthesis and Editing of 3D Shapes via Part-Aware Controlcited by 0
  3. 2026DiffGI: Differentiable Geometry Images for High-Fidelity Thin-Shell 3D Generationcited by 0
  4. 2026Learning Semantic Inpainting for Animatable Gaussian Head Avatarscited by 0
  5. 2026SPOON: Towards Coherent Compositional 3D Scene Generation from Uncalibrated Multi-view Imagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023GeoDream: Disentangling 2D and Geometric Priors for High-Fidelity and Consistent 3D Generation
  2. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models
  3. 2023PI3D: Efficient Text-to-3D Generation with Pseudo-Image Diffusion
  4. 2025Layout2Scene: 3D Semantic Layout Guided Scene Generation via Geometry and Appearance Diffusion Priors
  5. 2023Generating Images with 3D Annotations Using Diffusion Models
  6. 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generation
  7. 2023Breathing New Life into 3D Assets with Generative Repainting
  8. 2024A3D: Does Diffusion Dream about 3D Alignment?