# ANISE: Assembly-based Neural Implicit Surface rEconstruction

arXiv: https://arxiv.org/abs/2205.13682 · HTML: https://mlsanity.com/paper/2205.13682/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 62 references that are in the map.
- First posted on arXiv 27 May 2022.

## Builds on (most cited first, 30 of 34)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [Local Implicit Grid Representations for 3D Scenes](https://mlsanity.com/paper/2003.08981/) (2020)
- [DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction](https://mlsanity.com/paper/1905.10711/) (2019)
- [Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction](https://mlsanity.com/paper/2003.10983/) (2020)
- [What Do Single-view 3D Reconstruction Networks Learn?](https://mlsanity.com/paper/1905.03678/) (2019)
- [Learning Shape Templates with Structured Implicit Functions](https://mlsanity.com/paper/1904.06447/) (2019)
- [BSP-Net: Generating Compact Meshes via Binary Space Partitioning](https://mlsanity.com/paper/1911.06971/) (2019)
- [Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs](https://mlsanity.com/paper/1703.09438/) (2017)
- [Shape As Points: A Differentiable Poisson Solver](https://mlsanity.com/paper/2106.03452/) (2021)
- [Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistency](https://mlsanity.com/paper/1704.06254/) (2017)
- [CvxNet: Learnable Convex Decomposition](https://mlsanity.com/paper/1909.05736/) (2019)
- [Points2Surf: Learning Implicit Surfaces from Point Cloud Patches](https://mlsanity.com/paper/2007.10453/) (2020)
- [PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes](https://mlsanity.com/paper/1911.10949/) (2019)
- [Deep Geometric Prior for Surface Reconstruction](https://mlsanity.com/paper/1811.10943/) (2018)
- [Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks](https://mlsanity.com/paper/2103.10429/) (2021)
- [3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks](https://mlsanity.com/paper/1708.01648/) (2017)
- [Learning Shape Priors for Single-View 3D Completion and Reconstruction](https://mlsanity.com/paper/1809.05068/) (2018)
- [Learning to Infer and Execute 3D Shape Programs](https://mlsanity.com/paper/1901.02875/) (2019)
- [Im2Struct: Recovering 3D Shape Structure from a Single RGB Image](https://mlsanity.com/paper/1804.05469/) (2018)
- [Deep Meta Functionals for Shape Representation](https://mlsanity.com/paper/1908.06277/) (2019)
- [D2IM-Net: Learning Detail Disentangled Implicit Fields from Single Images](https://mlsanity.com/paper/2012.06650/) (2020)
- [Joint Learning of 3D Shape Retrieval and Deformation](https://mlsanity.com/paper/2101.07889/) (2021)
- [Learning Part Generation and Assembly for Structure-aware Shape Synthesis](https://mlsanity.com/paper/1906.06693/) (2019)

## Built on by (most cited first, 16 of 16)

- [PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers](https://mlsanity.com/paper/2506.05573/) (2025)
- [Part123: Part-aware 3D Reconstruction from a Single-view Image](https://mlsanity.com/paper/2405.16888/) (2024)
- [PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervision](https://mlsanity.com/paper/2303.09554/) (2023)
- [MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Clouds](https://mlsanity.com/paper/2508.14879/) (2025)
- [GEM3D: GEnerative Medial Abstractions for 3D Shape Synthesis](https://mlsanity.com/paper/2402.16994/) (2024)
- [1-Lipschitz Neural Distance Fields](https://mlsanity.com/paper/2407.09505/) (2024)
- [PartSDF: Part-Based Implicit Neural Representation for Composite 3D Shape Parametrization and Optimization](https://mlsanity.com/paper/2502.12985/) (2025)
- [GaussianBlock: Building Part-Aware Compositional and Editable 3D Scene by Primitives and Gaussians](https://mlsanity.com/paper/2410.01535/) (2024)
- [Part-aware Shape Generation with Latent 3D Diffusion of Neural Voxel Fields](https://mlsanity.com/paper/2405.00998/) (2024)
- [PSDF: Prior-Driven Neural Implicit Surface Learning for Multi-view Reconstruction](https://mlsanity.com/paper/2401.12751/) (2024)
- [MergeNet: Explicit Mesh Reconstruction from Sparse Point Clouds via Edge Prediction](https://mlsanity.com/paper/2407.11610/) (2024)
- [StructRe: Rewriting for Structured Shape Modeling](https://mlsanity.com/paper/2311.17510/) (2023)
- [Geometric Prior-Guided Neural Implicit Surface Reconstruction in the Wild](https://mlsanity.com/paper/2505.07373/) (2025)
- [S2MDF: A Plug-And-Play Layer for Intersection-Free Multi-Object Signed Distance Fields](https://mlsanity.com/paper/2605.29761/) (2026)
- [Diverse Part Synthesis for 3D Shape Creation](https://mlsanity.com/paper/2401.09384/) (2024)
- [SAR2Struct: Extracting 3D Semantic Structural Representation of Aircraft Targets from Single-View SAR Image](https://mlsanity.com/paper/2506.06757/) (2025)

## Abstract (arXiv)

We present ANISE, a method that reconstructs a 3D~shape from partial observations (images or sparse point clouds) using a part-aware neural implicit shape representation. The shape is formulated as an assembly of neural implicit functions, each representing a different part instance. In contrast to previous approaches, the prediction of this representation proceeds in a coarse-to-fine manner. Our model first reconstructs a structural arrangement of the shape in the form of geometric transformations of its part instances. Conditioned on them, the model predicts part latent codes encoding their surface geometry. Reconstructions can be obtained in two ways: (i) by directly decoding the part latent codes to part implicit functions, then combining them into the final shape; or (ii) by using part latents to retrieve similar part instances in a part database and assembling them in a single shape. We demonstrate that, when performing reconstruction by decoding part representations into implicit functions, our method achieves state-of-the-art part-aware reconstruction results from both images and sparse point clouds.When reconstructing shapes by assembling parts retrieved from a dataset, our approach significantly outperforms traditional shape retrieval methods even when significantly restricting the database size. We present our results in well-known sparse point cloud reconstruction and single-view reconstruction benchmarks.