# Improved Implicit Neural Representation with Fourier Reparameterized Training

arXiv: https://arxiv.org/abs/2401.07402 · HTML: https://mlsanity.com/paper/2401.07402/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 49 references that are in the map.
- First posted on arXiv 15 Jan 2024.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [RepVGG: Making VGG-style ConvNets Great Again](https://mlsanity.com/paper/2101.03697/) (2021)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019)
- [Local Implicit Grid Representations for 3D Scenes](https://mlsanity.com/paper/2003.08981/) (2020)
- [Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion](https://mlsanity.com/paper/2003.01456/) (2020)
- [Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction](https://mlsanity.com/paper/2003.10983/) (2020)
- [WIRE: Wavelet Implicit Neural Representations](https://mlsanity.com/paper/2301.05187/) (2023)
- [Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization](https://mlsanity.com/paper/1902.05967/) (2019)
- [ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks](https://mlsanity.com/paper/1908.03930/) (2019)
- [On the Inductive Bias of Neural Tangent Kernels](https://mlsanity.com/paper/1905.12173/) (2019)
- [The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies](https://mlsanity.com/paper/1906.00425/) (2019)
- [Beyond Periodicity: Towards a Unifying Framework for Activations in Coordinate-MLPs](https://mlsanity.com/paper/2111.15135/) (2021)
- [A Structured Dictionary Perspective on Implicit Neural Representations](https://mlsanity.com/paper/2112.01917/) (2021)
- [ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting](https://mlsanity.com/paper/2007.03260/) (2020)
- [FoV-NeRF: Foveated Neural Radiance Fields for Virtual Reality](https://mlsanity.com/paper/2103.16365/) (2021)
- [Deep Implicit Moving Least-Squares Functions for 3D Reconstruction](https://mlsanity.com/paper/2103.12266/) (2021)
- [Hypernetwork functional image representation](https://mlsanity.com/paper/1902.10404/) (2019)
- [Reconstructing continuous distributions of 3D protein structure from cryo-EM images](https://mlsanity.com/paper/1909.05215/) (2019)

## Built on by (most cited first, 30 of 36)

- [Where Do We Stand with Implicit Neural Representations? A Technical and Performance Survey](https://mlsanity.com/paper/2411.03688/) (2024)
- [Do We Always Need the Simplicity Bias? Looking for Optimal Inductive Biases in the Wild](https://mlsanity.com/paper/2503.10065/) (2025)
- [Inductive Gradient Adjustment For Spectral Bias In Implicit Neural Representations](https://mlsanity.com/paper/2410.13271/) (2024)
- [Implicit Neural Representations with Fourier Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2409.09323/) (2024)
- [Continuous Representation Methods, Theories, and Applications: An Overview and Perspectives](https://mlsanity.com/paper/2505.15222/) (2025)
- [PDEfuncta: Spectrally-Aware Neural Representation for PDE Solution Modeling](https://mlsanity.com/paper/2506.12790/) (2025)
- [SASNet: Spatially-Adaptive Sinusoidal Networks for INRs](https://mlsanity.com/paper/2503.09750/) (2025)
- [Cross-Frequency Implicit Neural Representation with Self-Evolving Parameters](https://mlsanity.com/paper/2504.10929/) (2025)
- [Calibrated Harmonic Overlaid Implicit Neural Representations for Multi-Dimensional Data](https://mlsanity.com/paper/2606.26763/) (2026)
- [MetricGrids: Arbitrary Nonlinear Approximation with Elementary Metric Grids based Implicit Neural Representation](https://mlsanity.com/paper/2503.10000/) (2025)
- [LIFT: Latent Implicit Functions for Task- and Data-Agnostic Encoding](https://mlsanity.com/paper/2503.15420/) (2025)
- [Exploring Kernel Transformations for Implicit Neural Representations](https://mlsanity.com/paper/2504.04728/) (2025)
- [Fourier-Modulated Implicit Neural Representation for Multispectral Satellite Image Compression](https://mlsanity.com/paper/2506.01234/) (2025)
- [Learning Pixel-adaptive Multi-layer Perceptrons for Real-time Image Enhancement](https://mlsanity.com/paper/2507.12135/) (2025)
- [Spectral Bottleneck in Sinusoidal Representation Networks: Noise is All You Need](https://mlsanity.com/paper/2509.09719/) (2025)
- [FM-SIREN & FM-FINER: Implicit Neural Representation Using Nyquist-based Orthogonality](https://mlsanity.com/paper/2509.23438/) (2025)
- [SeCo-INR: Semantically Conditioned Implicit Neural Representations for Improved Medical Image Super-Resolution](https://mlsanity.com/paper/2409.01013/) (2024)
- [QFGN: A Quantum Approach to High-Fidelity Implicit Neural Representations](https://mlsanity.com/paper/2504.19053/) (2025)
- [COSMO-INR: Complex Sinusoidal Modulation for Implicit Neural Representations](https://mlsanity.com/paper/2505.11640/) (2025)
- [Neu-PiG: Neural Preconditioned Grids for Fast Dynamic Surface Reconstruction on Long Sequences](https://mlsanity.com/paper/2602.22212/) (2026)
- [Reparameterized Tensor Ring Functional Decomposition for Multi-Dimensional Data Recovery](https://mlsanity.com/paper/2603.01034/) (2026)
- [Implicit Neural Representations: A Signal Processing Perspective](https://mlsanity.com/paper/2604.15047/) (2026)
- [SL2A-INR: Single-Layer Learnable Activation for Implicit Neural Representation](https://mlsanity.com/paper/2409.10836/) (2024)
- [LFR-PINO: A Layered Fourier Reduced Physics-Informed Neural Operator for Parametric PDEs](https://mlsanity.com/paper/2506.17582/) (2025)
- [FLAIR: Frequency- and Locality-Aware Implicit Neural Representations](https://mlsanity.com/paper/2508.13544/) (2025)
- [Adaptive Training of INRs via Pruning and Densification](https://mlsanity.com/paper/2510.23943/) (2025)
- [Neural Operator-Grounded Continuous Tensor Function Representation and Its Applications](https://mlsanity.com/paper/2603.01812/) (2026)
- [Coordinate Encoding on Linear Grids for Physics-Informed Neural Networks](https://mlsanity.com/paper/2603.22700/) (2026)
- [Neural Fields for NV-Center Inverse Sensing](https://mlsanity.com/paper/2605.13988/) (2026)
- [ImplicitTerrainV2: Wavelet-Guided Spatially Adaptive Neural Terrain Representation](https://mlsanity.com/paper/2605.22556/) (2026)

## Abstract (arXiv)

Implicit Neural Representation (INR) as a mighty representation paradigm has achieved success in various computer vision tasks recently. Due to the low-frequency bias issue of vanilla multi-layer perceptron (MLP), existing methods have investigated advanced techniques, such as positional encoding and periodic activation function, to improve the accuracy of INR. In this paper, we connect the network training bias with the reparameterization technique and theoretically prove that weight reparameterization could provide us a chance to alleviate the spectral bias of MLP. Based on our theoretical analysis, we propose a Fourier reparameterization method which learns coefficient matrix of fixed Fourier bases to compose the weights of MLP. We evaluate the proposed Fourier reparameterization method on different INR tasks with various MLP architectures, including vanilla MLP, MLP with positional encoding and MLP with advanced activation function, etc. The superiority approximation results on different MLP architectures clearly validate the advantage of our proposed method. Armed with our Fourier reparameterization method, better INR with more textures and less artifacts can be learned from the training data.