# Neural reproducing kernel Banach spaces and representer theorems for deep networks

arXiv: https://arxiv.org/abs/2403.08750 · HTML: https://mlsanity.com/paper/2403.08750/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Builds on (most cited first, 17 of 17)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [On Lazy Training in Differentiable Programming](https://mlsanity.com/paper/1812.07956/) (2018)
- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [On the Inductive Bias of Neural Tangent Kernels](https://mlsanity.com/paper/1905.12173/) (2019)
- [When Do Neural Networks Outperform Kernel Methods?](https://mlsanity.com/paper/2006.13409/) (2020)
- [A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case](https://mlsanity.com/paper/1910.01635/) (2019)
- [How do infinite width bounded norm networks look in function space?](https://mlsanity.com/paper/1902.05040/) (2019)
- [Deep Equals Shallow for ReLU Networks in Kernel Regimes](https://mlsanity.com/paper/2009.14397/) (2020)
- [What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory](https://mlsanity.com/paper/2105.03361/) (2021)
- [Understanding neural networks with reproducing kernel Banach spaces](https://mlsanity.com/paper/2109.09710/) (2021)
- [Random Neural Networks in the Infinite Width Limit as Gaussian Processes](https://mlsanity.com/paper/2107.01562/) (2021)
- [On Reproducing Kernel Banach Spaces: Generic Definitions and Unified Framework of Constructions](https://mlsanity.com/paper/1901.01002/) (2019)
- [Sparsity of solutions for variational inverse problems with finite-dimensional data](https://mlsanity.com/paper/1809.05045/) (2018)
- [A unifying representer theorem for inverse problems and machine learning](https://mlsanity.com/paper/1903.00687/) (2019)
- [Ridges, Neural Networks, and the Radon Transform](https://mlsanity.com/paper/2203.02543/) (2022)
- [From Kernel Methods to Neural Networks: A Unifying Variational Formulation](https://mlsanity.com/paper/2206.14625/) (2022)
- [Native Banach spaces for splines and variational inverse problems](https://mlsanity.com/paper/1904.10818/) (2019)

## Built on by (most cited first, 8 of 8)

- [A Lipschitz spaces view of infinitely wide shallow neural networks](https://mlsanity.com/paper/2410.14591/) (2024)
- [Vector-Valued Reproducing Kernel Banach Spaces for Neural Networks and Operators](https://mlsanity.com/paper/2509.26371/) (2025)
- [Representation Costs in Data Science: Foundations and the Quasi-Banach Spaces of Deep Neural Networks](https://mlsanity.com/paper/2606.14954/) (2026)
- [Brownian Kernel Ladders](https://mlsanity.com/paper/2606.15812/) (2026)
- [DeepMartingale: Duality of the Optimal Stopping Problem with Expressivity and High-Dimensional Hedging](https://mlsanity.com/paper/2510.13868/) (2025)
- [Deep Neural Variation Spaces: A Unifying Perspective on Depth and Complexity](https://mlsanity.com/paper/2607.05546/) (2026)
- [Variation Brownian Kernel Ladders](https://mlsanity.com/paper/2608.13882/) (2026)
- [Piecewise Linear Equivariant Maps for Compact Groups](https://mlsanity.com/paper/2608.21645/) (2026)

## Abstract (arXiv)

Characterizing the function spaces defined by neural networks helps understanding the corresponding learning models and their inductive bias. While in some limits neural networks correspond to function spaces that are Hilbert spaces, these regimes do not capture the properties of the networks used in practice. Indeed, several results have shown that shallow networks can be better characterized in terms of suitable Banach spaces. However, analogous results for deep networks are limited. In this paper we show that deep neural networks define suitable reproducing kernel Banach spaces. These spaces are equipped with norms that enforce a form of sparsity, enabling them to adapt to potential latent structures within the input data and their representations. In particular, by leveraging the theory of reproducing kernel Banach spaces, combined with variational results, we derive representer theorems that justify the finite architectures commonly employed in applications. Our study extends analogous results for shallow networks and represents a step towards understanding the function spaces induced by neural architectures used in practice.