# Smaller generalization error derived for a deep residual neural network compared to shallow networks

arXiv: https://arxiv.org/abs/2010.01887 · HTML: https://mlsanity.com/paper/2010.01887/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2020.

## Built on by (most cited first, 1 of 1)

- [A Qualitative Test-Risk Mechanism for Scaling Behavior in Normalized Residual Networks](https://mlsanity.com/paper/2605.08297/) (2026)

## Abstract (arXiv)

Estimates of the generalization error are proved for a residual neural network with L random Fourier features layers bar zℓ+1=bar zℓ + Resumk=1Kbar bℓ ke^iωℓ kbar zℓ+ Resumk=1Kbar cℓ ke^iω'ℓ k· x. An optimal distribution for the frequencies (ωℓ k,ω'ℓ k) of the random Fourier features e^iωℓ kbar zℓ and e^iω'ℓ k· x is derived. This derivation is based on the corresponding generalization error for the approximation of the function values f(x). The generalization error turns out to be smaller than the estimate |hat f|2L1(ℝd)/(KL) of the generalization error for random Fourier features with one hidden layer and the same total number of nodes KL, in the case the L^∞-norm of f is much less than the L1-norm of its Fourier transform hat f. This understanding of an optimal distribution for random features is used to construct a new training method for a deep residual network. Promising performance of the proposed new algorithm is demonstrated in computational experiments.