# On the exact computation of linear frequency principle dynamics and its generalization

arXiv: https://arxiv.org/abs/2010.08153 · HTML: https://mlsanity.com/paper/2010.08153/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2020.

## Built on by (most cited first, 16 of 16)

- [Overview frequency principle/spectral bias in deep learning](https://mlsanity.com/paper/2201.07395/) (2022)
- [On the Activation Function Dependence of the Spectral Bias of Neural Networks](https://mlsanity.com/paper/2208.04924/) (2022)
- [Spectral Bias Outside the Training Set for Deep Networks in the Kernel Regime](https://mlsanity.com/paper/2206.02927/) (2022)
- [Subspace Decomposition based DNN algorithm for elliptic type multi-scale PDEs](https://mlsanity.com/paper/2112.06660/) (2021)
- [How You Start Matters for Generalization](https://mlsanity.com/paper/2206.08558/) (2022)
- [Frequency Principle in Deep Learning Beyond Gradient-descent-based Training](https://mlsanity.com/paper/2101.00747/) (2021)
- [An Upper Limit of Decaying Rate with Respect to Frequency in Deep Neural Network](https://mlsanity.com/paper/2105.11675/) (2021)
- [Understanding the dynamics of the frequency bias in neural networks](https://mlsanity.com/paper/2405.14957/) (2024)
- [A rationale from frequency perspective for grokking in training neural network](https://mlsanity.com/paper/2405.17479/) (2024)
- [FW-GAN: Frequency-Driven Handwriting Synthesis with Wave-Modulated MLP Generator](https://mlsanity.com/paper/2508.21040/) (2025)
- [Limitation of Characterizing Implicit Regularization by Data-independent Functions](https://mlsanity.com/paper/2201.12198/) (2022)
- [Loss Jump During Loss Switch in Solving PDEs with Neural Networks](https://mlsanity.com/paper/2405.03095/) (2024)
- [Pseudo-differential-enhanced physics-informed neural networks](https://mlsanity.com/paper/2602.14663/) (2026)
- [Shearlet Neural Operators for Anisotropic-Shock-Dominated and Multi-scale parametric partial differential equations](https://mlsanity.com/paper/2604.25181/) (2026)
- [Deciphering Neural Reparameterized Full-Waveform Inversion with Neural Sensitivity Kernel and Wave Tangent Kernel](https://mlsanity.com/paper/2605.14370/) (2026)
- [SpiS-GAN: Spiral-Modulated Handwriting Synthesis with Star Operation](https://mlsanity.com/paper/2607.06949/) (2026)

## Abstract (arXiv)

Recent works show an intriguing phenomenon of Frequency Principle (F-Principle) that deep neural networks (DNNs) fit the target function from low to high frequency during the training, which provides insight into the training and generalization behavior of DNNs in complex tasks. In this paper, through analysis of an infinite-width two-layer NN in the neural tangent kernel (NTK) regime, we derive the exact differential equation, namely Linear Frequency-Principle (LFP) model, governing the evolution of NN output function in the frequency domain during the training. Our exact computation applies for general activation functions with no assumption on size and distribution of training data. This LFP model unravels that higher frequencies evolve polynomially or exponentially slower than lower frequencies depending on the smoothness/regularity of the activation function. We further bridge the gap between training dynamics and generalization by proving that LFP model implicitly minimizes a Frequency-Principle norm (FP-norm) of the learned function, by which higher frequencies are more severely penalized depending on the inverse of their evolution rate. Finally, we derive an textita priori generalization error bound controlled by the FP-norm of the target function, which provides a theoretical justification for the empirical results that DNNs often generalize well for low frequency functions.