# Deep Implicit Moving Least-Squares Functions for 3D Reconstruction

arXiv: https://arxiv.org/abs/2103.12266 · HTML: https://mlsanity.com/paper/2103.12266/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 60 references that are in the map.
- First posted on arXiv 23 Mar 2021.

## Builds on (most cited first, 30 of 37)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [Deep Learning for 3D Point Clouds: A Survey](https://mlsanity.com/paper/1912.12033/) (2019)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019)
- [Local Implicit Grid Representations for 3D Scenes](https://mlsanity.com/paper/2003.08981/) (2020)
- [Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion](https://mlsanity.com/paper/2003.01456/) (2020)
- [OctNet: Learning Deep 3D Representations at High Resolutions](https://mlsanity.com/paper/1611.05009/) (2016)
- [Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction](https://mlsanity.com/paper/2003.10983/) (2020)
- [What Do Single-view 3D Reconstruction Networks Learn?](https://mlsanity.com/paper/1905.03678/) (2019)
- [Differentiable Surface Splatting for Point-based Geometry Processing](https://mlsanity.com/paper/1906.04173/) (2019)
- [PU-Net: Point Cloud Upsampling Network](https://mlsanity.com/paper/1801.06761/) (2018)
- [Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs](https://mlsanity.com/paper/1703.09438/) (2017)
- [Learning Shape Abstractions by Assembling Volumetric Primitives](https://mlsanity.com/paper/1612.00404/) (2016)
- [Generative and Discriminative Voxel Modeling with Convolutional Neural Networks](https://mlsanity.com/paper/1608.04236/) (2016)
- [Deep Geometric Prior for Surface Reconstruction](https://mlsanity.com/paper/1811.10943/) (2018)
- [Patch-based Progressive 3D Point Set Upsampling](https://mlsanity.com/paper/1811.11286/) (2018)
- [Learning elementary structures for 3D shape generation and matching](https://mlsanity.com/paper/1908.04725/) (2019)

## Built on by (most cited first, 30 of 67)

- [Neural Wavelet-domain Diffusion for 3D Shape Generation](https://mlsanity.com/paper/2209.08725/) (2022)
- [ShapeFormer: Transformer-based Shape Completion via Sparse Representation](https://mlsanity.com/paper/2201.10326/) (2022)
- [Neural Kernel Surface Reconstruction](https://mlsanity.com/paper/2305.19590/) (2023)
- [SDF-StyleGAN: Implicit SDF-Based StyleGAN for 3D Shape Generation](https://mlsanity.com/paper/2206.12055/) (2022)
- [Dual Octree Graph Networks for Learning Adaptive Volumetric Shape Representations](https://mlsanity.com/paper/2205.02825/) (2022)
- [Towards Implicit Text-Guided 3D Shape Generation](https://mlsanity.com/paper/2203.14622/) (2022)
- [Physical Adversarial Attack meets Computer Vision: A Decade Survey](https://mlsanity.com/paper/2209.15179/) (2022)
- [Reconstructing Surfaces for Sparse Point Clouds with On-Surface Priors](https://mlsanity.com/paper/2204.10603/) (2022)
- [Surface Reconstruction from Point Clouds by Learning Predictive Context Priors](https://mlsanity.com/paper/2204.11015/) (2022)
- [CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimization](https://mlsanity.com/paper/2210.02757/) (2022)
- [Learning Deep Implicit Functions for 3D Shapes with Dynamic Code Clouds](https://mlsanity.com/paper/2203.14048/) (2022)
- [Surface Reconstruction from Point Clouds: A Survey and a Benchmark](https://mlsanity.com/paper/2205.02413/) (2022)
- [Towards Better Gradient Consistency for Neural Signed Distance Functions via Level Set Alignment](https://mlsanity.com/paper/2305.11601/) (2023)
- [Improved Implicit Neural Representation with Fourier Reparameterized Training](https://mlsanity.com/paper/2401.07402/) (2024)
- [GeoUDF: Surface Reconstruction from 3D Point Clouds via Geometry-guided Distance Representation](https://mlsanity.com/paper/2211.16762/) (2022)
- [Unsupervised Inference of Signed Distance Functions from Single Sparse Point Clouds without Learning Priors](https://mlsanity.com/paper/2303.14505/) (2023)
- [Neural Wavelet-domain Diffusion for 3D Shape Generation, Inversion, and Manipulation](https://mlsanity.com/paper/2302.00190/) (2023)
- [Learning Signed Distance Functions from Noisy 3D Point Clouds via Noise to Noise Mapping](https://mlsanity.com/paper/2306.01405/) (2023)
- [OctField: Hierarchical Implicit Functions for 3D Modeling](https://mlsanity.com/paper/2111.01067/) (2021)
- [Few 'Zero Level Set'-Shot Learning of Shape Signed Distance Functions in Feature Space](https://mlsanity.com/paper/2207.04161/) (2022)
- [SP-GAN: Sphere-Guided 3D Shape Generation and Manipulation](https://mlsanity.com/paper/2108.04476/) (2021)
- [Latent Partition Implicit with Surface Codes for 3D Representation](https://mlsanity.com/paper/2207.08631/) (2022)
- [VisCo Grids: Surface Reconstruction with Viscosity and Coarea Grids](https://mlsanity.com/paper/2303.14569/) (2023)
- [Unsupervised Occupancy Learning from Sparse Point Cloud](https://mlsanity.com/paper/2404.02759/) (2024)
- [Fast Learning of Signed Distance Functions from Noisy Point Clouds via Noise to Noise Mapping](https://mlsanity.com/paper/2407.14225/) (2024)
- [Neural-IMLS: Self-supervised Implicit Moving Least-Squares Network for Surface Reconstruction](https://mlsanity.com/paper/2109.04398/) (2021)
- [DeepCurrents: Learning Implicit Representations of Shapes with Boundaries](https://mlsanity.com/paper/2111.09383/) (2021)
- [Mixing-Denoising Generalizable Occupancy Networks](https://mlsanity.com/paper/2311.12125/) (2023)
- [PoNQ: a Neural QEM-based Mesh Representation](https://mlsanity.com/paper/2403.12870/) (2024)
- [Laplacian2Mesh: Laplacian-Based Mesh Understanding](https://mlsanity.com/paper/2202.00307/) (2022)

## Abstract (arXiv)

Point set is a flexible and lightweight representation widely used for 3D deep learning. However, their discrete nature prevents them from representing continuous and fine geometry, posing a major issue for learning-based shape generation. In this work, we turn the discrete point sets into smooth surfaces by introducing the well-known implicit moving least-squares (IMLS) surface formulation, which naturally defines locally implicit functions on point sets. We incorporate IMLS surface generation into deep neural networks for inheriting both the flexibility of point sets and the high quality of implicit surfaces. Our IMLSNet predicts an octree structure as a scaffold for generating MLS points where needed and characterizes shape geometry with learned local priors. Furthermore, our implicit function evaluation is independent of the neural network once the MLS points are predicted, thus enabling fast runtime evaluation. Our experiments on 3D object reconstruction demonstrate that IMLSNets outperform state-of-the-art learning-based methods in terms of reconstruction quality and computational efficiency. Extensive ablation tests also validate our network design and loss functions.