# Learning to Infer Implicit Surfaces without 3D Supervision

arXiv: https://arxiv.org/abs/1911.00767 · HTML: https://mlsanity.com/paper/1911.00767/

## Key facts

- Cited by 103 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2019.

## Built on by (most cited first, 30 of 103)

- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs](https://mlsanity.com/paper/2112.00724/) (2021)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [3D Neural Field Generation using Triplane Diffusion](https://mlsanity.com/paper/2211.16677/) (2022)
- [Editing Conditional Radiance Fields](https://mlsanity.com/paper/2105.06466/) (2021)
- [NEAT: Neural Attention Fields for End-to-End Autonomous Driving](https://mlsanity.com/paper/2109.04456/) (2021)
- [GNeRF: GAN-based Neural Radiance Field without Posed Camera](https://mlsanity.com/paper/2103.15606/) (2021)
- [Efficient Neural Radiance Fields for Interactive Free-viewpoint Video](https://mlsanity.com/paper/2112.01517/) (2021)
- [Neural Wavelet-domain Diffusion for 3D Shape Generation](https://mlsanity.com/paper/2209.08725/) (2022)
- [NeILF: Neural Incident Light Field for Physically-based Material Estimation](https://mlsanity.com/paper/2203.07182/) (2022)
- [Generative Neural Articulated Radiance Fields](https://mlsanity.com/paper/2206.14314/) (2022)
- [POCO: Point Convolution for Surface Reconstruction](https://mlsanity.com/paper/2201.01831/) (2022)
- [4K4D: Real-Time 4D View Synthesis at 4K Resolution](https://mlsanity.com/paper/2310.11448/) (2023)
- [Learning Signed Distance Field for Multi-view Surface Reconstruction](https://mlsanity.com/paper/2108.09964/) (2021)
- [Removing Objects From Neural Radiance Fields](https://mlsanity.com/paper/2212.11966/) (2022)
- [Shelf-Supervised Mesh Prediction in the Wild](https://mlsanity.com/paper/2102.06195/) (2021)
- [CAPRI-Net: Learning Compact CAD Shapes with Adaptive Primitive Assembly](https://mlsanity.com/paper/2104.05652/) (2021)
- [SAPE: Spatially-Adaptive Progressive Encoding for Neural Optimization](https://mlsanity.com/paper/2104.09125/) (2021)
- [CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fields](https://mlsanity.com/paper/2103.17269/) (2021)
- [Im4D: High-Fidelity and Real-Time Novel View Synthesis for Dynamic Scenes](https://mlsanity.com/paper/2310.08585/) (2023)
- [Implicit Neural Video Compression](https://mlsanity.com/paper/2112.11312/) (2021)
- [Critical Regularizations for Neural Surface Reconstruction in the Wild](https://mlsanity.com/paper/2206.03087/) (2022)
- [Seeing Implicit Neural Representations as Fourier Series](https://mlsanity.com/paper/2109.00249/) (2021)
- [A Skeleton-Driven Neural Occupancy Representation for Articulated Hands](https://mlsanity.com/paper/2109.11399/) (2021)
- [Implicit Autoencoder for Point-Cloud Self-Supervised Representation Learning](https://mlsanity.com/paper/2201.00785/) (2022)
- [Aug-NeRF: Training Stronger Neural Radiance Fields with Triple-Level Physically-Grounded Augmentations](https://mlsanity.com/paper/2207.01164/) (2022)
- [Multi-View Mesh Reconstruction with Neural Deferred Shading](https://mlsanity.com/paper/2212.04386/) (2022)
- [NeUDF: Leaning Neural Unsigned Distance Fields with Volume Rendering](https://mlsanity.com/paper/2304.10080/) (2023)

## Abstract (arXiv)

Recent advances in 3D deep learning have shown that it is possible to train highly effective deep models for 3D shape generation, directly from 2D images. This is particularly interesting since the availability of 3D models is still limited compared to the massive amount of accessible 2D images, which is invaluable for training. The representation of 3D surfaces itself is a key factor for the quality and resolution of the 3D output. While explicit representations, such as point clouds and voxels, can span a wide range of shape variations, their resolutions are often limited. Mesh-based representations are more efficient but are limited by their ability to handle varying topologies. Implicit surfaces, however, can robustly handle complex shapes, topologies, and also provide flexible resolution control. We address the fundamental problem of learning implicit surfaces for shape inference without the need of 3D supervision. Despite their advantages, it remains nontrivial to (1) formulate a differentiable connection between implicit surfaces and their 2D renderings, which is needed for image-based supervision; and (2) ensure precise geometric properties and control, such as local smoothness. In particular, sampling implicit surfaces densely is also known to be a computationally demanding and very slow operation. To this end, we propose a novel ray-based field probing technique for efficient image-to-field supervision, as well as a general geometric regularizer for implicit surfaces, which provides natural shape priors in unconstrained regions. We demonstrate the effectiveness of our framework on the task of single-view image-based 3D shape digitization and show how we outperform state-of-the-art techniques both quantitatively and qualitatively.