# POCO: Point Convolution for Surface Reconstruction

arXiv: https://arxiv.org/abs/2201.01831 · HTML: https://mlsanity.com/paper/2201.01831/

## Key facts

- Cited by 92 papers in the map (data through 14 Oct 2026).
- Builds on 93 of its 141 references that are in the map.
- First posted on arXiv 5 Jan 2022.

## Builds on (most cited first, 30 of 93)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019)

## Built on by (most cited first, 30 of 92)

- [3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models](https://mlsanity.com/paper/2301.11445/) (2023)
- [3D Neural Field Generation using Triplane Diffusion](https://mlsanity.com/paper/2211.16677/) (2022)
- [Neural Kernel Surface Reconstruction](https://mlsanity.com/paper/2305.19590/) (2023)
- [GSDF: 3DGS Meets SDF for Improved Rendering and Reconstruction](https://mlsanity.com/paper/2403.16964/) (2024)
- [ALSO: Automotive Lidar Self-supervision by Occupancy estimation](https://mlsanity.com/paper/2212.05867/) (2022)
- [Reconstructing Surfaces for Sparse Point Clouds with On-Surface Priors](https://mlsanity.com/paper/2204.10603/) (2022)
- [Surface Reconstruction from Point Clouds by Learning Predictive Context Priors](https://mlsanity.com/paper/2204.11015/) (2022)
- [Neural Signed Distance Function Inference through Splatting 3D Gaussians Pulled on Zero-Level Set](https://mlsanity.com/paper/2410.14189/) (2024)
- [CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimization](https://mlsanity.com/paper/2210.02757/) (2022)
- [Surface Reconstruction from Point Clouds: A Survey and a Benchmark](https://mlsanity.com/paper/2205.02413/) (2022)
- [Towards Better Gradient Consistency for Neural Signed Distance Functions via Level Set Alignment](https://mlsanity.com/paper/2305.11601/) (2023)
- [Learning a More Continuous Zero Level Set in Unsigned Distance Fields through Level Set Projection](https://mlsanity.com/paper/2308.11441/) (2023)
- [Neural-Singular-Hessian: Implicit Neural Representation of Unoriented Point Clouds by Enforcing Singular Hessian](https://mlsanity.com/paper/2309.01793/) (2023)
- [GeoUDF: Surface Reconstruction from 3D Point Clouds via Geometry-guided Distance Representation](https://mlsanity.com/paper/2211.16762/) (2022)
- [Unsupervised Inference of Signed Distance Functions from Single Sparse Point Clouds without Learning Priors](https://mlsanity.com/paper/2303.14505/) (2023)
- [SE(3)-Equivariant Attention Networks for Shape Reconstruction in Function Space](https://mlsanity.com/paper/2204.02394/) (2022)
- [Learning Signed Distance Functions from Noisy 3D Point Clouds via Noise to Noise Mapping](https://mlsanity.com/paper/2306.01405/) (2023)
- [Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Driving](https://mlsanity.com/paper/2311.11056/) (2023)
- [Latent Partition Implicit with Surface Codes for 3D Representation](https://mlsanity.com/paper/2207.08631/) (2022)
- [ALTO: Alternating Latent Topologies for Implicit 3D Reconstruction](https://mlsanity.com/paper/2212.04096/) (2022)
- [SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation](https://mlsanity.com/paper/2304.03251/) (2023)
- [SpaceMesh: A Continuous Representation for Learning Manifold Surface Meshes](https://mlsanity.com/paper/2409.20562/) (2024)
- [SparseCraft: Few-Shot Neural Reconstruction through Stereopsis Guided Geometric Linearization](https://mlsanity.com/paper/2407.14257/) (2024)
- [Unsupervised Occupancy Learning from Sparse Point Cloud](https://mlsanity.com/paper/2404.02759/) (2024)
- [Fast Learning of Signed Distance Functions from Noisy Point Clouds via Noise to Noise Mapping](https://mlsanity.com/paper/2407.14225/) (2024)
- [TINC: Tree-structured Implicit Neural Compression](https://mlsanity.com/paper/2211.06689/) (2022)
- [ZeroGrasp: Zero-Shot Shape Reconstruction Enabled Robotic Grasping](https://mlsanity.com/paper/2504.10857/) (2025)
- [Mixing-Denoising Generalizable Occupancy Networks](https://mlsanity.com/paper/2311.12125/) (2023)
- [NeurCADRecon: Neural Representation for Reconstructing CAD Surfaces by Enforcing Zero Gaussian Curvature](https://mlsanity.com/paper/2404.13420/) (2024)
- [Points2NeRF: Generating Neural Radiance Fields from 3D point cloud](https://mlsanity.com/paper/2206.01290/) (2022)

## Abstract (arXiv)

Implicit neural networks have been successfully used for surface reconstruction from point clouds. However, many of them face scalability issues as they encode the isosurface function of a whole object or scene into a single latent vector. To overcome this limitation, a few approaches infer latent vectors on a coarse regular 3D grid or on 3D patches, and interpolate them to answer occupancy queries. In doing so, they loose the direct connection with the input points sampled on the surface of objects, and they attach information uniformly in space rather than where it matters the most, i.e., near the surface. Besides, relying on fixed patch sizes may require discretization tuning. To address these issues, we propose to use point cloud convolutions and compute latent vectors at each input point. We then perform a learning-based interpolation on nearest neighbors using inferred weights. Experiments on both object and scene datasets show that our approach significantly outperforms other methods on most classical metrics, producing finer details and better reconstructing thinner volumes. The code is available at https://github.com/valeoai/POCO.