# Deep Learning with Kernels through RKHM and the Perron-Frobenius Operator

arXiv: https://arxiv.org/abs/2305.13588 · HTML: https://mlsanity.com/paper/2305.13588/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 40 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 23 of 23)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [Benign Overfitting in Linear Regression](https://mlsanity.com/paper/1906.11300/) (2019)
- [Norm-Based Capacity Control in Neural Networks](https://mlsanity.com/paper/1503.00036/) (2015)
- [Size-Independent Sample Complexity of Neural Networks](https://mlsanity.com/paper/1712.06541/) (2017)
- [A vector-contraction inequality for Rademacher complexities](https://mlsanity.com/paper/1605.00251/) (2016)
- [Does data interpolation contradict statistical optimality?](https://mlsanity.com/paper/1806.09471/) (2018)
- [The Promises and Pitfalls of Deep Kernel Learning](https://mlsanity.com/paper/2102.12108/) (2021)
- [Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS](https://mlsanity.com/paper/2009.10683/) (2020)
- [Convolutional Kernel Networks](https://mlsanity.com/paper/1406.3332/) (2014)
- [Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network](https://mlsanity.com/paper/1909.11274/) (2019)
- [Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees](https://mlsanity.com/paper/2206.02659/) (2022)
- [A representer theorem for deep kernel learning](https://mlsanity.com/paper/1709.10441/) (2017)
- [Scalable Matrix-valued Kernel Learning for High-dimensional Nonlinear Multivariate Regression and Granger Causality](https://mlsanity.com/paper/1408.2066/) (2014)
- [Autoencoding any Data through Kernel Autoencoders](https://mlsanity.com/paper/1805.11028/) (2018)
- [Reproducing kernel Hilbert C*-module and kernel mean embeddings](https://mlsanity.com/paper/2101.11410/) (2021)
- [C^*-algebra Net: A New Approach Generalizing Neural Network Parameters to C^*-algebra](https://mlsanity.com/paper/2206.09513/) (2022)
- [Metric on Nonlinear Dynamical Systems with Perron-Frobenius Operators](https://mlsanity.com/paper/1805.12324/) (2018)
- [Krylov Subspace Method for Nonlinear Dynamical Systems with Random Noise](https://mlsanity.com/paper/1909.03634/) (2019)
- [Entangled Kernels -- Beyond Separability](https://mlsanity.com/paper/2101.05514/) (2021)
- [Learning in RKHM: a C^*-Algebraic Twist for Kernel Machines](https://mlsanity.com/paper/2210.11855/) (2022)
- [Noncommutative C^*-algebra Net: Learning Neural Networks with Powerful Product Structure in C^*-algebra](https://mlsanity.com/paper/2302.01191/) (2023)
- [Analysis via Orthonormal Systems in Reproducing Kernel Hilbert C^*-Modules and Applications](https://mlsanity.com/paper/2003.00738/) (2020)

## Built on by (most cited first, 7 of 7)

- [C^*-Algebraic Machine Learning: Moving in a New Direction](https://mlsanity.com/paper/2402.02637/) (2024)
- [Spectral Truncation Kernels: Noncommutativity in C^*-algebraic Kernel Machines](https://mlsanity.com/paper/2405.17823/) (2024)
- [Position: Quantum Kernel Machines Should Move Beyond Scalar-Valued Kernels to Realize Their Potential](https://mlsanity.com/paper/2506.03779/) (2025)
- [Operator-Based Generalization Bound for Deep Learning: Insights on Multi-Task Learning](https://mlsanity.com/paper/2512.19184/) (2025)
- [Quantum Circuit C^*-algebra Net](https://mlsanity.com/paper/2404.06218/) (2024)
- [Predicting AI Agent Behavior through Approximation of the Perron-Frobenius Operator](https://mlsanity.com/paper/2406.02723/) (2024)
- [Compact Vision Transformer by Reduction of Kernel Complexity](https://mlsanity.com/paper/2507.12780/) (2025)

## Abstract (arXiv)

Reproducing kernel Hilbert C^*-module (RKHM) is a generalization of reproducing kernel Hilbert space (RKHS) by means of C^*-algebra, and the Perron-Frobenius operator is a linear operator related to the composition of functions. Combining these two concepts, we present deep RKHM, a deep learning framework for kernel methods. We derive a new Rademacher generalization bound in this setting and provide a theoretical interpretation of benign overfitting by means of Perron-Frobenius operators. By virtue of C^*-algebra, the dependency of the bound on output dimension is milder than existing bounds. We show that C^*-algebra is a suitable tool for deep learning with kernels, enabling us to take advantage of the product structure of operators and to provide a clear connection with convolutional neural networks. Our theoretical analysis provides a new lens through which one can design and analyze deep kernel methods.