# Learning Smooth Neural Functions via Lipschitz Regularization

arXiv: https://arxiv.org/abs/2202.08345 · HTML: https://mlsanity.com/paper/2202.08345/

## Key facts

- Cited by 86 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 60 references that are in the map.
- First posted on arXiv 16 Feb 2022.

## Builds on (most cited first, 30 of 43)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [Lipschitz regularity of deep neural networks: analysis and efficient estimation](https://mlsanity.com/paper/1805.10965/) (2018)
- [Regularisation of Neural Networks by Enforcing Lipschitz Continuity](https://mlsanity.com/paper/1804.04368/) (2018)
- [Parseval Networks: Improving Robustness to Adversarial Examples](https://mlsanity.com/paper/1704.08847/) (2017)
- [Towards Fast Computation of Certified Robustness for ReLU Networks](https://mlsanity.com/paper/1804.09699/) (2018)
- [Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer](https://mlsanity.com/paper/1908.01210/) (2019)
- [Sorting out Lipschitz function approximation](https://mlsanity.com/paper/1811.05381/) (2018)
- [Spectral Norm Regularization for Improving the Generalizability of Deep Learning](https://mlsanity.com/paper/1705.10941/) (2017)
- [Robustness via curvature regularization, and vice versa](https://mlsanity.com/paper/1811.09716/) (2018)
- [Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks](https://mlsanity.com/paper/1907.04595/) (2019)
- [Robust Learning with Jacobian Regularization](https://mlsanity.com/paper/1908.02729/) (2019)
- [Deep Geometric Prior for Surface Reconstruction](https://mlsanity.com/paper/1811.10943/) (2018)
- [Exactly Computing the Local Lipschitz Constant of ReLU Networks](https://mlsanity.com/paper/2003.01219/) (2020)
- [Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks](https://mlsanity.com/paper/1709.06079/) (2017)

## Built on by (most cited first, 30 of 86)

- [MeshDiffusion: Score-based Generative 3D Mesh Modeling](https://mlsanity.com/paper/2303.08133/) (2023)
- [Input Perturbation Reduces Exposure Bias in Diffusion Models](https://mlsanity.com/paper/2301.11706/) (2023)
- [E-NeRV: Expedite Neural Video Representation with Disentangled Spatial-Temporal Context](https://mlsanity.com/paper/2207.08132/) (2022)
- [PermutoSDF: Fast Multi-View Reconstruction with Implicit Surfaces using Permutohedral Lattices](https://mlsanity.com/paper/2211.12562/) (2022)
- [Deep Learning on Implicit Neural Representations of Shapes](https://mlsanity.com/paper/2302.05438/) (2023)
- [SpotlessSplats: Ignoring Distractors in 3D Gaussian Splatting](https://mlsanity.com/paper/2406.20055/) (2024)
- [A Policy-Guided Imitation Approach for Offline Reinforcement Learning](https://mlsanity.com/paper/2210.08323/) (2022)
- [CROM: Continuous Reduced-Order Modeling of PDEs Using Implicit Neural Representations](https://mlsanity.com/paper/2206.02607/) (2022)
- [Direct Parameterization of Lipschitz-Bounded Deep Networks](https://mlsanity.com/paper/2301.11526/) (2023)
- [A Level Set Theory for Neural Implicit Evolution under Explicit Flows](https://mlsanity.com/paper/2204.07159/) (2022)
- [On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning](https://mlsanity.com/paper/2210.06443/) (2022)
- [Weight Clipping for Deep Continual and Reinforcement Learning](https://mlsanity.com/paper/2407.01704/) (2024)
- [IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracing](https://mlsanity.com/paper/2312.05210/) (2023)
- [DecisionNCE: Embodied Multimodal Representations via Implicit Preference Learning](https://mlsanity.com/paper/2402.18137/) (2024)
- [RoMo: Robust Motion Segmentation Improves Structure from Motion](https://mlsanity.com/paper/2411.18650/) (2024)
- [Advancing Bayesian Optimization via Learning Correlated Latent Space](https://mlsanity.com/paper/2310.20258/) (2023)
- [Neuroplastic Expansion in Deep Reinforcement Learning](https://mlsanity.com/paper/2410.07994/) (2024)
- [SDF4CHD: Generative Modeling of Cardiac Anatomies with Congenital Heart Defects](https://mlsanity.com/paper/2311.00332/) (2023)
- [Unsupervised Occupancy Learning from Sparse Point Cloud](https://mlsanity.com/paper/2404.02759/) (2024)
- [Transforming Radiance Field with Lipschitz Network for Photorealistic 3D Scene Stylization](https://mlsanity.com/paper/2303.13232/) (2023)
- [Rethinking Robustness Assessment: Adversarial Attacks on Learning-based Quadrupedal Locomotion Controllers](https://mlsanity.com/paper/2405.12424/) (2024)
- [Action Tokenizer Matters in In-Context Imitation Learning](https://mlsanity.com/paper/2503.01206/) (2025)
- [FaceCLIPNeRF: Text-driven 3D Face Manipulation using Deformable Neural Radiance Fields](https://mlsanity.com/paper/2307.11418/) (2023)
- [Mixing-Denoising Generalizable Occupancy Networks](https://mlsanity.com/paper/2311.12125/) (2023)
- [Geometry-Informed Neural Networks](https://mlsanity.com/paper/2402.14009/) (2024)
- [3D Reconstruction with Fast Dipole Sums](https://mlsanity.com/paper/2405.16788/) (2024)
- [Deep Deformable 3D Caricatures with Learned Shape Control](https://mlsanity.com/paper/2207.14593/) (2022)
- [Explicit Neural Surfaces: Learning Continuous Geometry With Deformation Fields](https://mlsanity.com/paper/2306.02956/) (2023)
- [Shape of my heart: Cardiac models through learned signed distance functions](https://mlsanity.com/paper/2308.16568/) (2023)
- [Robustifying Generalizable Implicit Shape Networks with a Tunable Non-Parametric Model](https://mlsanity.com/paper/2311.12967/) (2023)

## Abstract (arXiv)

Neural implicit fields have recently emerged as a useful representation for 3D shapes. These fields are commonly represented as neural networks which map latent descriptors and 3D coordinates to implicit function values. The latent descriptor of a neural field acts as a deformation handle for the 3D shape it represents. Thus, smoothness with respect to this descriptor is paramount for performing shape-editing operations. In this work, we introduce a novel regularization designed to encourage smooth latent spaces in neural fields by penalizing the upper bound on the field's Lipschitz constant. Compared with prior Lipschitz regularized networks, ours is computationally fast, can be implemented in four lines of code, and requires minimal hyperparameter tuning for geometric applications. We demonstrate the effectiveness of our approach on shape interpolation and extrapolation as well as partial shape reconstruction from 3D point clouds, showing both qualitative and quantitative improvements over existing state-of-the-art and non-regularized baselines.