# Tuning Frequency Bias in Neural Network Training with Nonuniform Data

arXiv: https://arxiv.org/abs/2205.14300 · HTML: https://mlsanity.com/paper/2205.14300/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- First posted on arXiv 28 May 2022.

## Abstract (arXiv)

Small generalization errors of over-parameterized neural networks (NNs) can be partially explained by the frequency biasing phenomenon, where gradient-based algorithms minimize the low-frequency misfit before reducing the high-frequency residuals. Using the Neural Tangent Kernel (NTK), one can provide a theoretically rigorous analysis for training where data are drawn from constant or piecewise-constant probability densities. Since most training data sets are not drawn from such distributions, we use the NTK model and a data-dependent quadrature rule to theoretically quantify the frequency biasing of NN training given fully nonuniform data. By replacing the loss function with a carefully selected Sobolev norm, we can further amplify, dampen, counterbalance, or reverse the intrinsic frequency biasing in NN training.