# Geometry-Consistent Neural Shape Representation with Implicit Displacement Fields

arXiv: https://arxiv.org/abs/2106.05187 · HTML: https://mlsanity.com/paper/2106.05187/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 62 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 30 of 38)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [Local Implicit Grid Representations for 3D Scenes](https://mlsanity.com/paper/2003.08981/) (2020)
- [Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction](https://mlsanity.com/paper/2003.10983/) (2020)
- [Learning Shape Templates with Structured Implicit Functions](https://mlsanity.com/paper/1904.06447/) (2019)
- [BSP-Net: Generating Compact Meshes via Binary Space Partitioning](https://mlsanity.com/paper/1911.06971/) (2019)
- [FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction](https://mlsanity.com/paper/2003.13989/) (2020)
- [Thingi10K: A Dataset of 10,000 3D-Printing Models](https://mlsanity.com/paper/1605.04797/) (2016)
- [Training behavior of deep neural network in frequency domain](https://mlsanity.com/paper/1807.01251/) (2018)
- [CvxNet: Learnable Convex Decomposition](https://mlsanity.com/paper/1909.05736/) (2019)
- [NASA: Neural Articulated Shape Approximation](https://mlsanity.com/paper/1912.03207/) (2019)
- [Frequency Bias in Neural Networks for Input of Non-Uniform Density](https://mlsanity.com/paper/2003.04560/) (2020)

## Built on by (most cited first, 30 of 60)

- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [BACON: Band-limited Coordinate Networks for Multiscale Scene Representation](https://mlsanity.com/paper/2112.04645/) (2021)
- [HF-NeuS: Improved Surface Reconstruction Using High-Frequency Details](https://mlsanity.com/paper/2206.07850/) (2022)
- [EVA3D: Compositional 3D Human Generation from 2D Image Collections](https://mlsanity.com/paper/2210.04888/) (2022)
- [DiGS : Divergence guided shape implicit neural representation for unoriented point clouds](https://mlsanity.com/paper/2106.10811/) (2021)
- [Advances in 3D Generation: A Survey](https://mlsanity.com/paper/2401.17807/) (2024)
- [Physics Informed Neural Fields for Smoke Reconstruction with Sparse Data](https://mlsanity.com/paper/2206.06577/) (2022)
- [Neural-Singular-Hessian: Implicit Neural Representation of Unoriented Point Clouds by Enforcing Singular Hessian](https://mlsanity.com/paper/2309.01793/) (2023)
- [GETAvatar: Generative Textured Meshes for Animatable Human Avatars](https://mlsanity.com/paper/2310.02714/) (2023)
- [Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projection Matching](https://mlsanity.com/paper/2108.03746/) (2021)
- [Residual Multiplicative Filter Networks for Multiscale Reconstruction](https://mlsanity.com/paper/2206.00746/) (2022)
- [Robust Zero Level-Set Extraction from Unsigned Distance Fields Based on Double Covering](https://mlsanity.com/paper/2310.03431/) (2023)
- [MINER: Multiscale Implicit Neural Representations](https://mlsanity.com/paper/2202.03532/) (2022)
- [Intrinsic Neural Fields: Learning Functions on Manifolds](https://mlsanity.com/paper/2203.07967/) (2022)
- [PINs: Progressive Implicit Networks for Multi-Scale Neural Representations](https://mlsanity.com/paper/2202.04713/) (2022)
- [Tuning the Frequencies: Robust Training for Sinusoidal Neural Networks](https://mlsanity.com/paper/2407.21121/) (2024)
- [NeuForm: Adaptive Overfitting for Neural Shape Editing](https://mlsanity.com/paper/2207.08890/) (2022)
- [Text2Scene: Text-driven Indoor Scene Stylization with Part-aware Details](https://mlsanity.com/paper/2308.16880/) (2023)
- [URHand: Universal Relightable Hands](https://mlsanity.com/paper/2401.05334/) (2024)
- [AvatarStudio: High-fidelity and Animatable 3D Avatar Creation from Text](https://mlsanity.com/paper/2311.17917/) (2023)
- [NeurCADRecon: Neural Representation for Reconstructing CAD Surfaces by Enforcing Zero Gaussian Curvature](https://mlsanity.com/paper/2404.13420/) (2024)
- [Spelunking the Deep: Guaranteed Queries on General Neural Implicit Surfaces via Range Analysis](https://mlsanity.com/paper/2202.02444/) (2022)
- [ImFace++: A Sophisticated Nonlinear 3D Morphable Face Model with Implicit Neural Representations](https://mlsanity.com/paper/2312.04028/) (2023)
- [Neural Convolutional Surfaces](https://mlsanity.com/paper/2204.02289/) (2022)
- [NeurCross: A Neural Approach to Computing Cross Fields for Quad Mesh Generation](https://mlsanity.com/paper/2405.13745/) (2024)
- [Neural Experts: Mixture of Experts for Implicit Neural Representations](https://mlsanity.com/paper/2410.21643/) (2024)
- [2S-UDF: A Novel Two-stage UDF Learning Method for Robust Non-watertight Model Reconstruction from Multi-view Images](https://mlsanity.com/paper/2303.15368/) (2023)
- [Deformable Model-Driven Neural Rendering for High-Fidelity 3D Reconstruction of Human Heads Under Low-View Settings](https://mlsanity.com/paper/2303.13855/) (2023)
- [NeuroGF: A Neural Representation for Fast Geodesic Distance and Path Queries](https://mlsanity.com/paper/2306.00658/) (2023)

## Abstract (arXiv)

We present implicit displacement fields, a novel representation for detailed 3D geometry. Inspired by a classic surface deformation technique, displacement mapping, our method represents a complex surface as a smooth base surface plus a displacement along the base's normal directions, resulting in a frequency-based shape decomposition, where the high frequency signal is constrained geometrically by the low frequency signal. Importantly, this disentanglement is unsupervised thanks to a tailored architectural design that has an innate frequency hierarchy by construction. We explore implicit displacement field surface reconstruction and detail transfer and demonstrate superior representational power, training stability and generalizability.