# Neural Tangent Kernels and Fisher Information Matrices for Simple ReLU Networks with Random Hidden Weights

arXiv: https://arxiv.org/abs/2507.18555 · HTML: https://mlsanity.com/paper/2507.18555/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 5 references that are in the map.
- First posted on arXiv 24 Jul 2025.

## Builds on (most cited first, 4 of 4)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [On the Inductive Bias of Neural Tangent Kernels](https://mlsanity.com/paper/1905.12173/) (2019)
- [Approximate Spectral Decomposition of Fisher Information Matrix for Simple ReLU Networks](https://mlsanity.com/paper/2111.15256/) (2021)
- [Approximating Simple ReLU Networks based on Spectral Decomposition of Fisher Information](https://mlsanity.com/paper/2505.17907/) (2025)

## Abstract (arXiv)

Fisher information matrices and neural tangent kernels (NTK) for 2-layer ReLU networks with random hidden weight are argued. We discuss the relation between both notions as a linear transformation and show that spectral decomposition of NTK with concrete forms of eigenfunctions with major eigenvalues. We also obtain an approximation formula of the functions presented by the 2-layer neural networks.