mlsanityData to 7 Oct 2026

An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion

arXiv 2408.03178 ↗First posted 6 Aug 2024cs.CV cs.GR cs.LGcited by 34

Not on the map. Why?

Abstract (arXiv)

We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pixel image, effectively converting complex 3D shapes into a more manageable 2D format. By doing so, we address the challenges of both geometric and semantic irregularity inherent in polygonal meshes. This method allows us to use image generation models, such as Diffusion Transformers, directly for 3D shape generation. Evaluated on the ABO dataset, our generated shapes with patch structures achieve point cloud FID comparable to recent 3D generative models, while naturally supporting PBR material generation.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 54 of its 78 references that are in the map. First posted on arXiv 6 Aug 2024.

Builds on

54 of 78 references.

References · 54

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Scalable Diffusion Models with Transformerscited by 5,931
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  7. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  8. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  9. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  10. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  11. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  12. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
38 more
  1. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  2. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  3. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 449
  4. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  5. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 444
  6. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  7. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  8. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
  9. 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 267
  10. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 254
  11. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 210
  12. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  13. 2019BSP-Net: Generating Compact Meshes via Binary Space Partitioningcited by 205
  14. 2020DiffusionNet: Discretization Agnostic Learning on Surfacescited by 163
  15. 2024MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformerscited by 140
  16. 2022Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functionscited by 133
  17. 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generationcited by 129
  18. 2023Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Modelscited by 122
  19. 2019SDM-NET: Deep Generative Network for Structured Deformable Meshcited by 113
  20. 2018Deep Geometric Prior for Surface Reconstructioncited by 110
  21. 2022ShapeFormer: Transformer-based Shape Completion via Sparse Representationcited by 106
  22. 2024BrepGen: A B-rep Generative Diffusion Model with Structured Latent Geometrycited by 101
  23. 2019Learning elementary structures for 3D shape generation and matchingcited by 98
  24. 2024MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstructioncited by 94
  25. 20223DILG: Irregular Latent Grids for 3D Generative Modelingcited by 92
  26. 2022SDF-StyleGAN: Implicit SDF-Based StyleGAN for 3D Shape Generationcited by 92
  27. 2018MeshCNN: A Network with an Edgecited by 69
  28. 2023PolyDiff: Generating 3D Polygonal Meshes with Diffusion Modelscited by 66
  29. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  30. 2024Advances in 3D Generation: A Surveycited by 62
  31. 2018Scan2Mesh: From Unstructured Range Scans to 3D Meshescited by 51
  32. 20213D Shape Generation with Grid-based Implicit Functionscited by 48
  33. 2023Mosaic-SDF for 3D Generative Modelscited by 48
  34. 2019Shape Reconstruction by Learning Differentiable Surface Representationscited by 37
  35. 2018Multi-directional Geodesic Neural Networks via Equivariant Convolutioncited by 32
  36. 2018Multi-chart Generative Surface Modelingcited by 26
  37. 2020Pix2Surf: Learning Parametric 3D Surface Models of Objects from Imagescited by 22
  38. 2020Better Patch Stitching for Parametric Surface Reconstructioncited by 21

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 18; 2026: 11.

34 papers.

Sort:

By area:

  1. 20243DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusioncited by 100
  2. 2024Meshtron: High-Fidelity, Artist-Like 3D Mesh Generation at Scalecited by 87
  3. 2024EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generationcited by 84
  4. 2025Efficient Part-level 3D Object Generation via Dual Volume Packingcited by 47
  5. 2025NuiScene: Exploring Efficient Generation of Unbounded Outdoor Scenescited by 19
  6. 2025LL3M: Large Language 3D Modelerscited by 18
  7. 2024Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with Image-Based Surface Representationcited by 17
  8. 2024Wavelet Latent Diffusion (Wala): Billion-Parameter 3D Generative Model with Compact Wavelet Encodingscited by 10
  9. 2025Zero-1-to-G: Taming Pretrained 2D Diffusion Model for Direct 3D Generationcited by 8
  10. 2025UVGS: Reimagining Unstructured 3D Gaussian Splatting using UV Mappingcited by 8
  11. 2025Disentangled Clothed Avatar Generation with Layered Representationcited by 7
  12. 2025Repurposing 2D Diffusion Models with Gaussian Atlas for 3D Generationcited by 7
22 more
  1. 2025Generative Human Geometry Distributioncited by 4
  2. 2025GarmageNet: A Multimodal Generative Framework for Sewing Pattern Design and Generic Garment Modelingcited by 4
  3. 2026MeshTailor: Cutting Seams via Generative Mesh Traversalcited by 4
  4. 2026MeshFlow: Mesh Generation with Equivariant Flow Matchingcited by 3
  5. 2025SuperCarver: Texture-Consistent 3D Geometry Super-Resolution for High-Fidelity Surface Detail Generationcited by 2
  6. 2025Text2VDM: Text to Vector Displacement Maps for Expressive and Interactive 3D Sculptingcited by 1
  7. 2025GenVDM: Generating Vector Displacement Maps From a Single Imagecited by 1
  8. 2025Hyper Diffusion Avatars: Dynamic Human Avatar Generation using Network Weight Space Diffusioncited by 1
  9. 2025Repurposing 2D Diffusion Models for 3D Shape Completioncited by 1
  10. 2026AnySurf: Any Surface Generation with Directed Edgecited by 1
  11. 2025Deeply Dual Supervised learning for melanoma recognitioncited by 0
  12. 2025Edge Detection for Organ Boundaries via Top Down Refinement and SubPixel Upsamplingcited by 0
  13. 2025DualResolution Residual Architecture with Artifact Suppression for Melanocytic Lesion Segmentationcited by 0
  14. 2025VesselRW: Weakly Supervised Subcutaneous Vessel Segmentation via Learned Random Walk Propagationcited by 0
  15. 2026SwiftTailor: Efficient 3D Garment Generation with Geometry Image Representationcited by 0
  16. 2026LACON: Training Text-to-Image Model from Uncurated Datacited by 0
  17. 2026Mat'ern Noise for Triangulation-Agnostic Flow Matching on Meshescited by 0
  18. 2026DiffGI: Differentiable Geometry Images for High-Fidelity Thin-Shell 3D Generationcited by 0
  19. 2026RADmesh: Remesh-Aware Mesh Deformationcited by 0
  20. 2026Revisiting Avatar-As-Image: High-Fidelity Registration is All You Needcited by 0
  21. 2026Learning Semantic Inpainting for Animatable Gaussian Head Avatarscited by 0
  22. 2026SPOON: Towards Coherent Compositional 3D Scene Generation from Uncalibrated Multi-view Imagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion
  2. 2024Boosting 3D Object Generation through PBR Materials
  3. 2023MeshDiffusion: Score-based Generative 3D Mesh Modeling
  4. 2025SPGen: Spherical Projection as Consistent and Flexible Representation for Single Image 3D Shape Generation
  5. 2025Kiss3DGen: Repurposing Image Diffusion Models for 3D Asset Generation
  6. 2026Rethinking 3D Shape Generation: Diffusion over Superquadrics
  7. 2025MeshGen: Generating PBR Textured Mesh with Render-Enhanced Auto-Encoder and Generative Data Augmentation
  8. 2024Enhancing Single Image to 3D Generation using Gaussian Splatting and Hybrid Diffusion Priors