# PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations

arXiv: https://arxiv.org/abs/2008.01639 · HTML: https://mlsanity.com/paper/2008.01639/

## Key facts

- Cited by 81 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Aug 2020.

## Built on by (most cited first, 30 of 81)

- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation](https://mlsanity.com/paper/2104.07645/) (2021)
- [POCO: Point Convolution for Surface Reconstruction](https://mlsanity.com/paper/2201.01831/) (2022)
- [FoV-NeRF: Foveated Neural Radiance Fields for Virtual Reality](https://mlsanity.com/paper/2103.16365/) (2021)
- [SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements](https://mlsanity.com/paper/2104.07660/) (2021)
- [Towards Implicit Text-Guided 3D Shape Generation](https://mlsanity.com/paper/2203.14622/) (2022)
- [LISA: Learning Implicit Shape and Appearance of Hands](https://mlsanity.com/paper/2204.01695/) (2022)
- [Advances in 3D Generation: A Survey](https://mlsanity.com/paper/2401.17807/) (2024)
- [Geometry-Consistent Neural Shape Representation with Implicit Displacement Fields](https://mlsanity.com/paper/2106.05187/) (2021)
- [Reconstructing Surfaces for Sparse Point Clouds with On-Surface Priors](https://mlsanity.com/paper/2204.10603/) (2022)
- [SA-ConvONet: Sign-Agnostic Optimization of Convolutional Occupancy Networks](https://mlsanity.com/paper/2105.03582/) (2021)
- [Surface Reconstruction from Point Clouds by Learning Predictive Context Priors](https://mlsanity.com/paper/2204.11015/) (2022)
- [CViT: Continuous Vision Transformer for Operator Learning](https://mlsanity.com/paper/2405.13998/) (2024)
- [CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimization](https://mlsanity.com/paper/2210.02757/) (2022)
- [Dynamic Surface Function Networks for Clothed Human Bodies](https://mlsanity.com/paper/2104.03978/) (2021)
- [Learning Deep Implicit Functions for 3D Shapes with Dynamic Code Clouds](https://mlsanity.com/paper/2203.14048/) (2022)
- [Surface Reconstruction from Point Clouds: A Survey and a Benchmark](https://mlsanity.com/paper/2205.02413/) (2022)
- [Towards Better Gradient Consistency for Neural Signed Distance Functions via Level Set Alignment](https://mlsanity.com/paper/2305.11601/) (2023)
- [Topology-Preserving Shape Reconstruction and Registration via Neural Diffeomorphic Flow](https://mlsanity.com/paper/2203.08652/) (2022)
- [GenSDF: Two-Stage Learning of Generalizable Signed Distance Functions](https://mlsanity.com/paper/2206.02780/) (2022)
- [Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking](https://mlsanity.com/paper/2401.06614/) (2024)
- [Unsupervised Inference of Signed Distance Functions from Single Sparse Point Clouds without Learning Priors](https://mlsanity.com/paper/2303.14505/) (2023)
- [SE(3)-Equivariant Attention Networks for Shape Reconstruction in Function Space](https://mlsanity.com/paper/2204.02394/) (2022)
- [Neural Shape Deformation Priors](https://mlsanity.com/paper/2210.05616/) (2022)
- [Learning Signed Distance Functions from Noisy 3D Point Clouds via Noise to Noise Mapping](https://mlsanity.com/paper/2306.01405/) (2023)
- [Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projection Matching](https://mlsanity.com/paper/2108.03746/) (2021)
- [Few 'Zero Level Set'-Shot Learning of Shape Signed Distance Functions in Feature Space](https://mlsanity.com/paper/2207.04161/) (2022)
- [PatchComplete: Learning Multi-Resolution Patch Priors for 3D Shape Completion on Unseen Categories](https://mlsanity.com/paper/2206.04916/) (2022)

## Abstract (arXiv)

Implicit surface representations, such as signed-distance functions, combined with deep learning have led to impressive models which can represent detailed shapes of objects with arbitrary topology. Since a continuous function is learned, the reconstructions can also be extracted at any arbitrary resolution. However, large datasets such as ShapeNet are required to train such models. In this paper, we present a new mid-level patch-based surface representation. At the level of patches, objects across different categories share similarities, which leads to more generalizable models. We then introduce a novel method to learn this patch-based representation in a canonical space, such that it is as object-agnostic as possible. We show that our representation trained on one category of objects from ShapeNet can also well represent detailed shapes from any other category. In addition, it can be trained using much fewer shapes, compared to existing approaches. We show several applications of our new representation, including shape interpolation and partial point cloud completion. Due to explicit control over positions, orientations and scales of patches, our representation is also more controllable compared to object-level representations, which enables us to deform encoded shapes non-rigidly.