# Deep Networks are Reproducing Kernel Chains

arXiv: https://arxiv.org/abs/2501.03697 · HTML: https://mlsanity.com/paper/2501.03697/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jan 2025.

## Built on by (most cited first, 8 of 8)

- [Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature Learning](https://mlsanity.com/paper/2605.19458/) (2026)
- [Vector-Valued Reproducing Kernel Banach Spaces for Neural Networks and Operators](https://mlsanity.com/paper/2509.26371/) (2025)
- [Representation Costs in Data Science: Foundations and the Quasi-Banach Spaces of Deep Neural Networks](https://mlsanity.com/paper/2606.14954/) (2026)
- [Brownian Kernel Ladders](https://mlsanity.com/paper/2606.15812/) (2026)
- [Deep Neural Variation Spaces: A Unifying Perspective on Depth and Complexity](https://mlsanity.com/paper/2607.05546/) (2026)
- [Variation Brownian Kernel Ladders](https://mlsanity.com/paper/2608.13882/) (2026)
- [Featured Reproducing Kernel Banach Spaces for Learning and Neural Networks](https://mlsanity.com/paper/2602.07141/) (2026)
- [Common Covariance Geometry and Certification for Brownian Kernel Ladders](https://mlsanity.com/paper/2609.29525/) (2026)

## Abstract (arXiv)

Identifying an appropriate function space for deep neural networks remains a key open question. While shallow neural networks are naturally associated with Reproducing Kernel Banach Spaces (RKBS), deep networks present unique challenges. In this work, we extend RKBS to chain RKBS (cRKBS), a new framework that composes kernels rather than functions, preserving the desirable properties of RKBS. We prove that any deep neural network function is a neural cRKBS function, and conversely, any neural cRKBS function defined on a finite dataset corresponds to a deep neural network. This approach provides a sparse solution to the empirical risk minimization problem, requiring no more than N neurons per layer, where N is the number of data points.