# On the Empirical Neural Tangent Kernel of Standard Finite-Width Convolutional Neural Network Architectures

arXiv: https://arxiv.org/abs/2006.13645 · HTML: https://mlsanity.com/paper/2006.13645/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2020.

## Built on by (most cited first, 3 of 3)

- [The Unreasonable Effectiveness of Patches in Deep Convolutional Kernels Methods](https://mlsanity.com/paper/2101.07528/) (2021)
- [Weighted Neural Tangent Kernel: A Generalized and Improved Network-Induced Kernel](https://mlsanity.com/paper/2103.11558/) (2021)
- [On the Equivalence of Random Network Distillation, Deep Ensembles, and Bayesian Inference](https://mlsanity.com/paper/2602.19964/) (2026)

## Abstract (arXiv)

The Neural Tangent Kernel (NTK) is an important milestone in the ongoing effort to build a theory for deep learning. Its prediction that sufficiently wide neural networks behave as kernel methods, or equivalently as random feature models, has been confirmed empirically for certain wide architectures. It remains an open question how well NTK theory models standard neural network architectures of widths common in practice, trained on complex datasets such as ImageNet. We study this question empirically for two well-known convolutional neural network architectures, namely AlexNet and LeNet, and find that their behavior deviates significantly from their finite-width NTK counterparts. For wider versions of these networks, where the number of channels and widths of fully-connected layers are increased, the deviation decreases.