mlsanityData to 7 Oct 2026

On Regularizing Coordinate-MLPs

arXiv 2202.00790 ↗First posted 1 Feb 2022cs.LG cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

We show that typical implicit regularization assumptions for deep neural networks (for regression) do not hold for coordinate-MLPs, a family of MLPs that are now ubiquitous in computer vision for representing high-frequency signals. Lack of such implicit bias disrupts smooth interpolations between training samples, and hampers generalizing across signal regions with different spectra. We investigate this behavior through a Fourier lens and uncover that as the bandwidth of a coordinate-MLP is enhanced, lower frequencies tend to get suppressed unless a suitable prior is provided explicitly. Based on these insights, we propose a simple regularization technique that can mitigate the above problem, which can be incorporated into existing networks without any architectural modifications.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 32 of its 48 references that are in the map. First posted on arXiv 1 Feb 2022.

Builds on

32 of 48 references.

References · 32

  1. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  2. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  3. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  4. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  5. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  6. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  7. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  8. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  9. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  10. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  11. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  12. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
20 more
  1. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 649
  2. 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 339
  3. 2018Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Datacited by 260
  4. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  5. 2020DeRF: Decomposed Radiance Fieldscited by 163
  6. 2019NASA: Neural Articulated Shape Approximationcited by 149
  7. 2021Beyond Periodicity: Towards a Unifying Framework for Activations in Coordinate-MLPscited by 134
  8. 2019Implicit Regularization of Discrete Gradient Dynamics in Linear Neural Networkscited by 122
  9. 2021A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representationcited by 113
  10. 2019How do infinite width bounded norm networks look in function space?cited by 105
  11. 2021Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothingcited by 102
  12. 2021NeuTex: Neural Texture Mapping for Volumetric Neural Renderingcited by 98
  13. 2018Gradient Descent Quantizes ReLU Network Featurescited by 57
  14. 2019Learning a Neural 3D Texture Space from 2D Exemplarscited by 53
  15. 2017Neural networks and rational functionscited by 40
  16. 2021Rethinking Positional Encodingcited by 36
  17. 2020On Polynomial Approximations for Privacy-Preserving and Verifiable ReLU Networkscited by 19
  18. 2019Implicit Regularization in Over-parameterized Neural Networkscited by 10
  19. 2021LightSAL: Lightweight Sign Agnostic Learning for Implicit Surface Representationcited by 7
  20. 2018Theory IIIb: Generalization in Deep Networkscited by 6

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 2.

5 papers.

  1. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  2. 2022GARF: Gaussian Activated Radiance Fields for High Fidelity Reconstruction and Pose Estimationcited by 86
  3. 2022E-NeRV: Expedite Neural Video Representation with Disentangled Spatial-Temporal Contextcited by 76
  4. 2023NeRFtrinsic Four: An End-To-End Trainable NeRF Jointly Optimizing Diverse Intrinsic and Extrinsic Camera Parameterscited by 6
  5. 2023Learning Lens Blur Fieldscited by 6