# Training Neural Networks Using Reproducing Kernel Space Interpolation and Model Reduction

arXiv: https://arxiv.org/abs/2308.16754 · HTML: https://mlsanity.com/paper/2308.16754/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 43 references that are in the map.
- First posted on arXiv 31 Aug 2023.

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problem](https://mlsanity.com/paper/2101.08286/) (2021)
- [Forward Thinking: Building and Training Neural Networks One Layer at a Time](https://mlsanity.com/paper/1706.02480/) (2017)
- [The universal approximation power of finite-width deep ReLU networks](https://mlsanity.com/paper/1806.01528/) (2018)
- [BPGrad: Towards Global Optimality in Deep Learning via Branch and Pruning](https://mlsanity.com/paper/1711.06959/) (2017)

## Built on by (most cited first, 1 of 1)

- [Kolmogorov-Arnold Fourier Networks](https://mlsanity.com/paper/2502.06018/) (2025)

## Abstract (arXiv)

We introduce and study the theory of training neural networks using interpolation techniques from reproducing kernel Hilbert space theory. We generalize the method to Krein spaces, and show that widely-used neural network architectures are subsets of reproducing kernel Krein spaces (RKKS). We study the concept of "associated Hilbert spaces" of RKKS and develop techniques to improve upon the expressivity of various activation functions. Next, using concepts from the theory of functions of several complex variables, we prove a computationally applicable, multidimensional generalization of the celebrated Adamjan- Arov-Krein (AAK) theorem. The theorem yields a novel class of neural networks, called Prolongation Neural Networks (PNN). We demonstrate that, by applying the multidimensional AAK theorem to gain a PNN, one can gain performance superior to both our interpolatory methods and current state-of-the-art methods in noisy environments. We provide useful illustrations of our methods in practice.