# GEM3D: GEnerative Medial Abstractions for 3D Shape Synthesis

arXiv: https://arxiv.org/abs/2402.16994 · HTML: https://mlsanity.com/paper/2402.16994/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 77 references that are in the map.
- First posted on arXiv 26 Feb 2024.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models](https://mlsanity.com/paper/2301.11445/) (2023)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation](https://mlsanity.com/paper/2212.04493/) (2022)
- [3D Neural Field Generation using Triplane Diffusion](https://mlsanity.com/paper/2211.16677/) (2022)
- [Thingi10K: A Dataset of 10,000 3D-Printing Models](https://mlsanity.com/paper/1605.04797/) (2016)
- [Neural Wavelet-domain Diffusion for 3D Shape Generation](https://mlsanity.com/paper/2209.08725/) (2022)
- [Locally Attentional SDF Diffusion for Controllable 3D Shape Generation](https://mlsanity.com/paper/2305.04461/) (2023)
- [PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes](https://mlsanity.com/paper/1911.10949/) (2019)
- [3DILG: Irregular Latent Grids for 3D Generative Modeling](https://mlsanity.com/paper/2205.13914/) (2022)
- [Scan2Mesh: From Unstructured Range Scans to 3D Meshes](https://mlsanity.com/paper/1811.10464/) (2018)
- [Predicting Animation Skeletons for 3D Articulated Models via Volumetric Nets](https://mlsanity.com/paper/1908.08506/) (2019)
- [A Skeleton-bridged Deep Learning Approach for Generating Meshes of Complex Topologies from Single RGB Images](https://mlsanity.com/paper/1903.04704/) (2019)
- [RigNet: Neural Rigging for Articulated Characters](https://mlsanity.com/paper/2005.00559/) (2020)
- [Composite Shape Modeling via Latent Space Factorization](https://mlsanity.com/paper/1901.02968/) (2019)
- [SkeletonNet: A Topology-Preserving Solution for Learning Mesh Reconstruction of Object Surfaces from RGB Images](https://mlsanity.com/paper/2008.05742/) (2020)
- [Skeleton-bridged Point Completion: From Global Inference to Local Adjustment](https://mlsanity.com/paper/2010.07428/) (2020)
- [Coverage Axis: Inner Point Selection for 3D Shape Skeletonization](https://mlsanity.com/paper/2110.00965/) (2021)

## Built on by (most cited first, 12 of 12)

- [Generative AI for Film Creation: A Survey of Recent Advances](https://mlsanity.com/paper/2504.08296/) (2025)
- [LaGeM: A Large Geometry Model for 3D Representation Learning and Diffusion](https://mlsanity.com/paper/2410.01295/) (2024)
- [PrimitiveAnything: Human-Crafted 3D Primitive Assembly Generation with Auto-Regressive Transformer](https://mlsanity.com/paper/2505.04622/) (2025)
- [Representing 3D Shapes With 64 Latent Vectors for 3D Diffusion Models](https://mlsanity.com/paper/2503.08737/) (2025)
- [TetWeave: Isosurface Extraction using On-The-Fly Delaunay Tetrahedral Grids for Gradient-Based Mesh Optimization](https://mlsanity.com/paper/2505.04590/) (2025)
- [CoFie: Learning Compact Neural Surface Representations with Coordinate Fields](https://mlsanity.com/paper/2406.03417/) (2024)
- [SOPHY: Learning to Generate Simulation-Ready Objects with Physical Materials](https://mlsanity.com/paper/2504.12684/) (2025)
- [CHARM: Control-point-based 3D Anime Hairstyle Auto-Regressive Modeling](https://mlsanity.com/paper/2509.21114/) (2025)
- [Rethinking Metrics and Diffusion Architecture for 3D Point Cloud Generation](https://mlsanity.com/paper/2511.05308/) (2025)
- [High-Fidelity Medical Shape Generation via Skeletal Latent Diffusion](https://mlsanity.com/paper/2603.07504/) (2026)
- [Geometry Distributions](https://mlsanity.com/paper/2411.16076/) (2024)
- [Rigel3D: Rig-aware Latents for Animation-Ready 3D Asset Generation](https://mlsanity.com/paper/2605.13129/) (2026)

## Abstract (arXiv)

We introduce GEM3D -- a new deep, topology-aware generative model of 3D shapes. The key ingredient of our method is a neural skeleton-based representation encoding information on both shape topology and geometry. Through a denoising diffusion probabilistic model, our method first generates skeleton-based representations following the Medial Axis Transform (MAT), then generates surfaces through a skeleton-driven neural implicit formulation. The neural implicit takes into account the topological and geometric information stored in the generated skeleton representations to yield surfaces that are more topologically and geometrically accurate compared to previous neural field formulations. We discuss applications of our method in shape synthesis and point cloud reconstruction tasks, and evaluate our method both qualitatively and quantitatively. We demonstrate significantly more faithful surface reconstruction and diverse shape generation results compared to the state-of-the-art, also involving challenging scenarios of reconstructing and synthesizing structurally complex, high-genus shape surfaces from Thingi10K and ShapeNet.