# 3D Shape Generation with Grid-based Implicit Functions

arXiv: https://arxiv.org/abs/2107.10607 · HTML: https://mlsanity.com/paper/2107.10607/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 35 references that are in the map.
- First posted on arXiv 22 Jul 2021.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [Local Implicit Grid Representations for 3D Scenes](https://mlsanity.com/paper/2003.08981/) (2020)
- [Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion](https://mlsanity.com/paper/2003.01456/) (2020)
- [3D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutions](https://mlsanity.com/paper/1905.06292/) (2019)
- [Stabilizing Training of Generative Adversarial Networks through Regularization](https://mlsanity.com/paper/1705.09367/) (2017)
- [Hierarchical Surface Prediction for 3D Object Reconstruction](https://mlsanity.com/paper/1704.00710/) (2017)
- [Point Cloud GAN](https://mlsanity.com/paper/1810.05795/) (2018)
- [Adversarial Generation of Continuous Implicit Shape Representations](https://mlsanity.com/paper/2002.00349/) (2020)
- [SurfNet: Generating 3D shape surfaces using deep residual networks](https://mlsanity.com/paper/1703.04079/) (2017)
- [A new graph-based two-sample test for multivariate and object data](https://mlsanity.com/paper/1307.6294/) (2013)
- [A Convolutional Decoder for Point Clouds using Adaptive Instance Normalization](https://mlsanity.com/paper/1906.11478/) (2019)
- [Gradient penalty from a maximum margin perspective](https://mlsanity.com/paper/1910.06922/) (2019)

## Built on by (most cited first, 30 of 48)

- [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models](https://mlsanity.com/paper/2301.11445/) (2023)
- [3D Neural Field Generation using Triplane Diffusion](https://mlsanity.com/paper/2211.16677/) (2022)
- [Neural Wavelet-domain Diffusion for 3D Shape Generation](https://mlsanity.com/paper/2209.08725/) (2022)
- [Locally Attentional SDF Diffusion for Controllable 3D Shape Generation](https://mlsanity.com/paper/2305.04461/) (2023)
- [Diffusion-SDF: Text-to-Shape via Voxelized Diffusion](https://mlsanity.com/paper/2212.03293/) (2022)
- [MeshXL: Neural Coordinate Field for Generative 3D Foundation Models](https://mlsanity.com/paper/2405.20853/) (2024)
- [SDF-StyleGAN: Implicit SDF-Based StyleGAN for 3D Shape Generation](https://mlsanity.com/paper/2206.12055/) (2022)
- [TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models](https://mlsanity.com/paper/2310.13772/) (2023)
- [Towards Implicit Text-Guided 3D Shape Generation](https://mlsanity.com/paper/2203.14622/) (2022)
- [SALAD: Part-Level Latent Diffusion for 3D Shape Generation and Manipulation](https://mlsanity.com/paper/2303.12236/) (2023)
- [ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representations](https://mlsanity.com/paper/2203.14510/) (2022)
- [CLIP-Sculptor: Zero-Shot Generation of High-Fidelity and Diverse Shapes from Natural Language](https://mlsanity.com/paper/2211.01427/) (2022)
- [Octree Transformer: Autoregressive 3D Shape Generation on Hierarchically Structured Sequences](https://mlsanity.com/paper/2111.12480/) (2021)
- [An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion](https://mlsanity.com/paper/2408.03178/) (2024)
- [Neural Wavelet-domain Diffusion for 3D Shape Generation, Inversion, and Manipulation](https://mlsanity.com/paper/2302.00190/) (2023)
- [Multi-modal Machine Learning in Engineering Design: A Review and Future Directions](https://mlsanity.com/paper/2302.10909/) (2023)
- [AI-Generated Content (AIGC) for Various Data Modalities: A Survey](https://mlsanity.com/paper/2308.14177/) (2023)
- [Make-A-Shape: a Ten-Million-scale 3D Shape Model](https://mlsanity.com/paper/2401.11067/) (2024)
- [LL3M: Large Language 3D Modelers](https://mlsanity.com/paper/2508.08228/) (2025)
- [EFEM: Equivariant Neural Field Expectation Maximization for 3D Object Segmentation Without Scene Supervision](https://mlsanity.com/paper/2303.15440/) (2023)
- [Pushing the Limits of 3D Shape Generation at Scale](https://mlsanity.com/paper/2306.11510/) (2023)
- [MinD-3D: Reconstruct High-quality 3D objects in Human Brain](https://mlsanity.com/paper/2312.07485/) (2023)
- [Humans as Light Bulbs: 3D Human Reconstruction from Thermal Reflection](https://mlsanity.com/paper/2305.01652/) (2023)
- [ImFace++: A Sophisticated Nonlinear 3D Morphable Face Model with Implicit Neural Representations](https://mlsanity.com/paper/2312.04028/) (2023)
- [A Survey on Computational Solutions for Reconstructing Complete Objects by Reassembling Their Fractured Parts](https://mlsanity.com/paper/2410.14770/) (2024)
- [CNS-Edit: 3D Shape Editing via Coupled Neural Shape Optimization](https://mlsanity.com/paper/2402.02313/) (2024)
- [Pushing Auto-regressive Models for 3D Shape Generation at Capacity and Scalability](https://mlsanity.com/paper/2402.12225/) (2024)
- [MinD-3D++: Advancing fMRI-Based 3D Reconstruction with High-Quality Textured Mesh Generation and a Comprehensive Dataset](https://mlsanity.com/paper/2409.11315/) (2024)

## Abstract (arXiv)

Previous approaches to generate shapes in a 3D setting train a GAN on the latent space of an autoencoder (AE). Even though this produces convincing results, it has two major shortcomings. As the GAN is limited to reproduce the dataset the AE was trained on, we cannot reuse a trained AE for novel data. Furthermore, it is difficult to add spatial supervision into the generation process, as the AE only gives us a global representation. To remedy these issues, we propose to train the GAN on grids (i.e. each cell covers a part of a shape). In this representation each cell is equipped with a latent vector provided by an AE. This localized representation enables more expressiveness (since the cell-based latent vectors can be combined in novel ways) as well as spatial control of the generation process (e.g. via bounding boxes). Our method outperforms the current state of the art on all established evaluation measures, proposed for quantitatively evaluating the generative capabilities of GANs. We show limitations of these measures and propose the adaptation of a robust criterion from statistical analysis as an alternative.