# Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks

arXiv: https://arxiv.org/abs/2103.10429 · HTML: https://mlsanity.com/paper/2103.10429/

## Key facts

- Cited by 103 papers in the map (data through 14 Oct 2026).
- Builds on 76 of its 98 references that are in the map.
- First posted on arXiv 18 Mar 2021.

## Builds on (most cited first, 30 of 76)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019)
- [Local Implicit Grid Representations for 3D Scenes](https://mlsanity.com/paper/2003.08981/) (2020)
- [Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion](https://mlsanity.com/paper/2003.01456/) (2020)
- [OctNet: Learning Deep 3D Representations at High Resolutions](https://mlsanity.com/paper/1611.05009/) (2016)
- [DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction](https://mlsanity.com/paper/1905.10711/) (2019)
- [Learning Category-Specific Mesh Reconstruction from Image Collections](https://mlsanity.com/paper/1803.07549/) (2018)
- [ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation](https://mlsanity.com/paper/1606.02147/) (2016)

## Built on by (most cited first, 30 of 103)

- [Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis](https://mlsanity.com/paper/2111.04276/) (2021)
- [ATISS: Autoregressive Transformers for Indoor Scene Synthesis](https://mlsanity.com/paper/2110.03675/) (2021)
- [Tracking Everything Everywhere All at Once](https://mlsanity.com/paper/2306.05422/) (2023)
- [GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Parts](https://mlsanity.com/paper/2211.05272/) (2022)
- [Flexible Isosurface Extraction for Gradient-Based Mesh Optimization](https://mlsanity.com/paper/2308.05371/) (2023)
- [SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements](https://mlsanity.com/paper/2104.07660/) (2021)
- [LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discovery](https://mlsanity.com/paper/2207.03434/) (2022)
- [SALAD: Part-Level Latent Diffusion for 3D Shape Generation and Manipulation](https://mlsanity.com/paper/2303.12236/) (2023)
- [LISA: Learning Implicit Shape and Appearance of Hands](https://mlsanity.com/paper/2204.01695/) (2022)
- [CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphism](https://mlsanity.com/paper/2203.16529/) (2022)
- [Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Camera](https://mlsanity.com/paper/2206.15258/) (2022)
- [Part123: Part-aware 3D Reconstruction from a Single-view Image](https://mlsanity.com/paper/2405.16888/) (2024)
- [D2CSG: Unsupervised Learning of Compact CSG Trees with Dual Complements and Dropouts](https://mlsanity.com/paper/2301.11497/) (2023)
- [Differentiable Blocks World: Qualitative 3D Decomposition by Rendering Primitives](https://mlsanity.com/paper/2307.05473/) (2023)
- [PartAfford: Part-level Affordance Discovery from 3D Objects](https://mlsanity.com/paper/2202.13519/) (2022)
- [PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervision](https://mlsanity.com/paper/2303.09554/) (2023)
- [Neural Template: Topology-aware Reconstruction and Disentangled Generation of 3D Meshes](https://mlsanity.com/paper/2206.04942/) (2022)
- [Hi-LASSIE: High-Fidelity Articulated Shape and Skeleton Discovery from Sparse Image Ensemble](https://mlsanity.com/paper/2212.11042/) (2022)
- [PartGlot: Learning Shape Part Segmentation from Language Reference Games](https://mlsanity.com/paper/2112.06390/) (2021)
- [Common Pets in 3D: Dynamic New-View Synthesis of Real-Life Deformable Categories](https://mlsanity.com/paper/2211.03889/) (2022)
- [Self-Supervised Category-Level Articulated Object Pose Estimation with Part-Level SE(3) Equivariance](https://mlsanity.com/paper/2302.14268/) (2023)
- [Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Images](https://mlsanity.com/paper/2204.09866/) (2022)
- [Latent Partition Implicit with Surface Codes for 3D Representation](https://mlsanity.com/paper/2207.08631/) (2022)
- [Point Scene Understanding via Disentangled Instance Mesh Reconstruction](https://mlsanity.com/paper/2203.16832/) (2022)
- [Marching-Primitives: Shape Abstraction from Signed Distance Function](https://mlsanity.com/paper/2303.13190/) (2023)
- [Coaching a Teachable Student](https://mlsanity.com/paper/2306.10014/) (2023)
- [Text2Scene: Text-driven Indoor Scene Stylization with Part-aware Details](https://mlsanity.com/paper/2308.16880/) (2023)
- [PRIF: Primary Ray-based Implicit Function](https://mlsanity.com/paper/2208.06143/) (2022)
- [NIVeL: Neural Implicit Vector Layers for Text-to-Vector Generation](https://mlsanity.com/paper/2405.15217/) (2024)
- [Go with the Flows: Mixtures of Normalizing Flows for Point Cloud Generation and Reconstruction](https://mlsanity.com/paper/2106.03135/) (2021)

## Abstract (arXiv)

Impressive progress in 3D shape extraction led to representations that can capture object geometries with high fidelity. In parallel, primitive-based methods seek to represent objects as semantically consistent part arrangements. However, due to the simplicity of existing primitive representations, these methods fail to accurately reconstruct 3D shapes using a small number of primitives/parts. We address the trade-off between reconstruction quality and number of parts with Neural Parts, a novel 3D primitive representation that defines primitives using an Invertible Neural Network (INN) which implements homeomorphic mappings between a sphere and the target object. The INN allows us to compute the inverse mapping of the homeomorphism, which in turn, enables the efficient computation of both the implicit surface function of a primitive and its mesh, without any additional post-processing. Our model learns to parse 3D objects into semantically consistent part arrangements without any part-level supervision. Evaluations on ShapeNet, D-FAUST and FreiHAND demonstrate that our primitives can capture complex geometries and thus simultaneously achieve geometrically accurate as well as interpretable reconstructions using an order of magnitude fewer primitives than state-of-the-art shape abstraction methods.