# Label-Aware Neural Tangent Kernel: Toward Better Generalization and Local Elasticity

arXiv: https://arxiv.org/abs/2010.11775 · HTML: https://mlsanity.com/paper/2010.11775/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2020.

## Built on by (most cited first, 18 of 18)

- [Neural Networks as Kernel Learners: The Silent Alignment Effect](https://mlsanity.com/paper/2111.00034/) (2021)
- [Neural Tangent Kernel Beyond the Infinite-Width Limit: Effects of Depth and Initialization](https://mlsanity.com/paper/2202.00553/) (2022)
- [Spectral Evolution and Invariance in Linear-width Neural Networks](https://mlsanity.com/paper/2211.06506/) (2022)
- [On the Generalization Mystery in Deep Learning](https://mlsanity.com/paper/2203.10036/) (2022)
- [Neural Tangent Kernel: A Survey](https://mlsanity.com/paper/2208.13614/) (2022)
- [Neural (Tangent Kernel) Collapse](https://mlsanity.com/paper/2305.16427/) (2023)
- [Finite-time Lyapunov exponents of deep neural networks](https://mlsanity.com/paper/2306.12548/) (2023)
- [Generalization Through The Lens Of Leave-One-Out Error](https://mlsanity.com/paper/2203.03443/) (2022)
- [Elephant Neural Networks: Born to Be a Continual Learner](https://mlsanity.com/paper/2310.01365/) (2023)
- [Uniform Convergence, Adversarial Spheres and a Simple Remedy](https://mlsanity.com/paper/2105.03491/) (2021)
- [Imitating Deep Learning Dynamics via Locally Elastic Stochastic Differential Equations](https://mlsanity.com/paper/2110.05960/) (2021)
- [Three Mechanisms of Feature Learning in a Linear Network](https://mlsanity.com/paper/2401.07085/) (2024)
- [Envisioning Future Deep Learning Theories: Some Basic Concepts and Characteristics](https://mlsanity.com/paper/2112.09741/) (2021)
- [Class-wise Generalization Error: an Information-Theoretic Analysis](https://mlsanity.com/paper/2401.02904/) (2024)
- [GradPCA: Leveraging NTK Alignment for Reliable Out-of-Distribution Detection](https://mlsanity.com/paper/2505.16017/) (2025)
- [Weighted Neural Tangent Kernel: A Generalized and Improved Network-Induced Kernel](https://mlsanity.com/paper/2103.11558/) (2021)
- [Dynamics of Local Elasticity During Training of Neural Nets](https://mlsanity.com/paper/2111.01166/) (2021)
- [Efficient Reinforcement Learning by Reducing Forgetting with Elephant Activation Functions](https://mlsanity.com/paper/2509.19159/) (2025)

## Abstract (arXiv)

As a popular approach to modeling the dynamics of training overparametrized neural networks (NNs), the neural tangent kernels (NTK) are known to fall behind real-world NNs in generalization ability. This performance gap is in part due to the textitlabel agnostic nature of the NTK, which renders the resulting kernel not as textitlocally elastic as NNs~citephe2019local. In this paper, we introduce a novel approach from the perspective of label-awareness to reduce this gap for the NTK. Specifically, we propose two label-aware kernels that are each a superimposition of a label-agnostic part and a hierarchy of label-aware parts with increasing complexity of label dependence, using the Hoeffding decomposition. Through both theoretical and empirical evidence, we show that the models trained with the proposed kernels better simulate NNs in terms of generalization ability and local elasticity.