# Sobolev Training for Implicit Neural Representations with Approximated Image Derivatives

arXiv: https://arxiv.org/abs/2207.10395 · HTML: https://mlsanity.com/paper/2207.10395/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 45 references that are in the map.
- First posted on arXiv 21 Jul 2022.

## Builds on (most cited first, 30 of 30)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering](https://mlsanity.com/paper/2106.02634/) (2021)
- [Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction](https://mlsanity.com/paper/2003.10983/) (2020)
- [Deep Learning for Single Image Super-Resolution: A Brief Review](https://mlsanity.com/paper/1808.03344/) (2018)
- [Sobolev Training for Neural Networks](https://mlsanity.com/paper/1706.04859/) (2017)
- [NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild](https://mlsanity.com/paper/2110.07604/) (2021)
- [A Structured Dictionary Perspective on Implicit Neural Representations](https://mlsanity.com/paper/2112.01917/) (2021)
- [Learning Signed Distance Field for Multi-view Surface Reconstruction](https://mlsanity.com/paper/2108.09964/) (2021)
- [Natural and Realistic Single Image Super-Resolution with Explicit Natural Manifold Discrimination](https://mlsanity.com/paper/1911.03624/) (2019)

## Built on by (most cited first, 6 of 6)

- [Accurate Differential Operators for Hybrid Neural Fields](https://mlsanity.com/paper/2312.05984/) (2023)
- [Training Image Derivatives: Increased Accuracy and Universal Robustness](https://mlsanity.com/paper/2310.14045/) (2023)
- [Dynamical Implicit Neural Representations](https://mlsanity.com/paper/2511.21787/) (2025)
- [ImplicitTerrainV2: Wavelet-Guided Spatially Adaptive Neural Terrain Representation](https://mlsanity.com/paper/2605.22556/) (2026)
- [Level-Crossing Density as a Mesh-Free High-Frequency Auxiliary Loss for Implicit Neural Representations](https://mlsanity.com/paper/2607.05815/) (2026)
- [Coarse-to-fine Framework for Generative MEF via Implicit Neural Representation](https://mlsanity.com/paper/2607.17611/) (2026)

## Abstract (arXiv)

Recently, Implicit Neural Representations (INRs) parameterized by neural networks have emerged as a powerful and promising tool to represent different kinds of signals due to its continuous, differentiable properties, showing superiorities to classical discretized representations. However, the training of neural networks for INRs only utilizes input-output pairs, and the derivatives of the target output with respect to the input, which can be accessed in some cases, are usually ignored. In this paper, we propose a training paradigm for INRs whose target output is image pixels, to encode image derivatives in addition to image values in the neural network. Specifically, we use finite differences to approximate image derivatives. We show how the training paradigm can be leveraged to solve typical INRs problems, i.e., image regression and inverse rendering, and demonstrate this training paradigm can improve the data-efficiency and generalization capabilities of INRs. The code of our method is available at urlhttps://github.com/megvii-research/SobolevINRs.