mlsanityData to 7 Oct 2026

Steps Toward Deep Kernel Methods from Infinite Neural Networks

arXiv 1508.05133 ↗First posted 20 Aug 2015cs.LG cs.NEcited by 28

Not on the map. Why?

Abstract (arXiv)

Contemporary deep neural networks exhibit impressive results on practical problems. These networks generalize well although their inherent capacity may extend significantly beyond the number of training examples. We analyze this behavior in the context of deep, infinite neural networks. We show that deep infinite layers are naturally aligned with Gaussian processes and kernel methods, and devise stochastic kernels that encode the information of these networks. We show that stability results apply despite the size, offering an explanation for their empirical success.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Aug 2015.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 12; 2022: 5; 2023: 4; 2024: 6; 2025: 1.

28 papers.

Sort:

  1. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  2. 2021Tensor Programs IIb: Architectural Universality of Neural Tangent Kernel Training Dynamicscited by 57
  3. 2021Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shapingcited by 51
  4. 2022A Fast, Well-Founded Approximation to the Empirical Neural Tangent Kernelcited by 30
  5. 2021Batch Normalization Orthogonalizes Representations in Deep Random Networkscited by 28
  6. 2021Non-asymptotic approximations of neural networks by Gaussian processescited by 18
  7. 2021Deep Stable neural networks: large-width asymptotics and convergence ratescited by 8
  8. 2021Imitating Deep Learning Dynamics via Locally Elastic Stochastic Differential Equationscited by 8
  9. 2024SHAP values via sparse Fourier representationcited by 8
  10. 2023Centered Self-Attention Layerscited by 7
  11. 2024NTK-Guided Few-Shot Class Incremental Learningcited by 7
  12. 2023Connecting NTK and NNGP: A Unified Theoretical Framework for Wide Neural Network Learning Dynamicscited by 6
16 more
  1. 2023Neural Hilbert Ladders: Multi-Layer Neural Networks in Function Spacecited by 5
  2. 2024Descriptive Kernel Convolution Network with Improved Random Walk Kernelcited by 5
  3. 2024Deep Horseshoe Gaussian Processescited by 5
  4. 2022Learning Representation from Neural Fisher Kernel with Low-rank Approximationcited by 4
  5. 2022Optimal Activation Functions for the Random Features Regression Modelcited by 4
  6. 2021Correlated Weights in Infinite Limits of Deep Convolutional Neural Networkscited by 3
  7. 2022Bayesian Neural Network Language Modeling for Speech Recognitioncited by 3
  8. 2024Neural networks in non-metric spacescited by 3
  9. 2024Finite Neural Networks as Mixtures of Gaussian Processes: From Provable Error Bounds to Prior Selectioncited by 3
  10. 2025MoSE: Unveiling Structural Patterns in Graphs via Mixture of Subgraph Expertscited by 3
  11. 2021Infinitely Wide Tensor Networks as Gaussian Processcited by 2
  12. 2021Implicit Acceleration and Feature Learning in Infinitely Wide Neural Networks with Bottleneckscited by 2
  13. 2022A connection between probability, physics and neural networkscited by 2
  14. 2021Weighted Neural Tangent Kernel: A Generalized and Improved Network-Induced Kernelcited by 1
  15. 2021Neural Generalization of Multiple Kernel Learningcited by 0
  16. 2023A Simple Algorithm For Scaling Up Kernel Methodscited by 0