mlsanityData to 7 Oct 2026

LaGeM: A Large Geometry Model for 3D Representation Learning and Diffusion

arXiv 2410.01295 ↗First posted 2 Oct 2024cs.CV cs.GRcited by 16

Not on the map. Why?

Abstract (arXiv)

This paper introduces a novel hierarchical autoencoder that maps 3D models into a highly compressed latent space. The hierarchical autoencoder is specifically designed to tackle the challenges arising from large-scale datasets and generative modeling using diffusion. Different from previous approaches that only work on a regular image or volume grid, our hierarchical autoencoder operates on unordered sets of vectors. Each level of the autoencoder controls different geometric levels of detail. We show that the model can be used to represent a wide range of 3D models while faithfully representing high-resolution geometry details. The training of the new architecture takes 0.70x time and 0.58x memory compared to the baseline. We also explore how the new representation can be used for generative modeling. Specifically, we propose a cascaded diffusion framework where each stage is conditioned on the previous stage. Our design extends existing cascaded designs for image and volume grids to vector sets.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 51 of its 56 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

51 of 56 references.

References · 51

  1. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  2. 2022Scalable Diffusion Models with Transformerscited by 5,931
  3. 2016Layer Normalizationcited by 5,579
  4. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  7. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  8. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  9. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  10. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  11. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  12. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
38 more
  1. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  2. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  3. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  4. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  5. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  6. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  7. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  8. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 449
  9. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 444
  10. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  11. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  12. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  13. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
  14. 20223D Neural Field Generation using Triplane Diffusioncited by 274
  15. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
  16. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 254
  17. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 225
  18. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  19. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 213
  20. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  21. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 183
  22. 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
  23. 2024MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformerscited by 140
  24. 2022Neural Wavelet-domain Diffusion for 3D Shape Generationcited by 132
  25. 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generationcited by 129
  26. 2022ShapeFormer: Transformer-based Shape Completion via Sparse Representationcited by 106
  27. 2024MeshXL: Neural Coordinate Field for Generative 3D Foundation Modelscited by 100
  28. 20243DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusioncited by 100
  29. 20223DILG: Irregular Latent Grids for 3D Generative Modelingcited by 92
  30. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 91
  31. 2020EGAD! an Evolved Grasping Analysis Dataset for diversity and reproducibility in robotic manipulationcited by 61
  32. 2023Mosaic-SDF for 3D Generative Modelscited by 48
  33. 2024OctFusion: Octree-based Diffusion Models for 3D Shape Generationcited by 40
  34. 2020ManifoldPlus: A Robust and Scalable Watertight Manifold Surface Generation Method for Triangle Soupscited by 38
  35. 2024Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Trackingcited by 27
  36. 20233DQD: Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Processcited by 25
  37. 2024MVD2: Efficient Multiview 3D Reconstruction for Multiview Diffusioncited by 22
  38. 2023Functional Diffusioncited by 13

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 10; 2026: 5.

16 papers.

Sort:

By area:

  1. 2024Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoderscited by 93
  2. 2025Sat2City: 3D City Generation from A Single Satellite Image with Cascaded Latent Diffusioncited by 15
  3. 2025iFlame: Interleaving Full and Linear Attention for Efficient Mesh Generationcited by 13
  4. 2025Hyper3D: Efficient 3D Representation via Hybrid Triplane and Octree Feature for Enhanced 3D Shape Variational Auto-Encoderscited by 7
  5. 2025Representing 3D Shapes With 64 Latent Vectors for 3D Diffusion Modelscited by 6
  6. 2025DiMeR: Disentangled Mesh Reconstruction Modelcited by 6
  7. 2025SOPHY: Learning to Generate Simulation-Ready Objects with Physical Materialscited by 5
  8. 2025Autoregressive Generation of Static and Growing Treescited by 3
  9. 2025GarmentX: Autoregressive Parametric Representations for High-Fidelity 3D Garment Generationcited by 3
  10. 2026TopoMesh: High-Fidelity Mesh Autoencoding via Topological Unificationcited by 2
  11. 2026Sat2City v2: Native 3D City Asset Generation from a Single Satellite Imagecited by 2
  12. 2025Memorization in 3D Shape Generation: An Empirical Studycited by 1
4 more
  1. 2026High-Fidelity Medical Shape Generation via Skeletal Latent Diffusioncited by 1
  2. 2025NoiseSDF2NoiseSDF: Learning Clean Neural Fields from Noisy Supervisioncited by 0
  3. 2026Generative Shape Reconstruction with Geometry-Guided Langevin Dynamicscited by 0
  4. 2026ZipTok3D: High-Fidelity 3D Tokenization with Compact Token Prefixescited by 0