# Mirror Descent on Reproducing Kernel Banach Spaces

arXiv: https://arxiv.org/abs/2411.11242 · HTML: https://mlsanity.com/paper/2411.11242/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 67 references that are in the map.
- First posted on arXiv 18 Nov 2024.

## Builds on (most cited first, 21 of 21)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Condition](https://mlsanity.com/paper/1608.04636/) (2016)
- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case](https://mlsanity.com/paper/1910.01635/) (2019)
- [How do infinite width bounded norm networks look in function space?](https://mlsanity.com/paper/1902.05040/) (2019)
- [A Priori Estimates of the Population Risk for Two-layer Neural Networks](https://mlsanity.com/paper/1810.06397/) (2018)
- [What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory](https://mlsanity.com/paper/2105.03361/) (2021)
- [Approximation spaces of deep neural networks](https://mlsanity.com/paper/1905.01208/) (2019)
- [Understanding neural networks with reproducing kernel Banach spaces](https://mlsanity.com/paper/2109.09710/) (2021)
- [Stochastic Mirror Descent on Overparameterized Nonlinear Models: Convergence, Implicit Regularization, and Generalization](https://mlsanity.com/paper/1906.03830/) (2019)
- [On Reproducing Kernel Banach Spaces: Generic Definitions and Unified Framework of Constructions](https://mlsanity.com/paper/1901.01002/) (2019)
- [Primal-Attention: Self-attention through Asymmetric Kernel SVD in Primal Representation](https://mlsanity.com/paper/2305.19798/) (2023)
- [Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently](https://mlsanity.com/paper/2205.12808/) (2022)
- [Transformers are Deep Infinite-Dimensional Non-Mercer Binary Kernel Machines](https://mlsanity.com/paper/2106.01506/) (2021)
- [Duality for Neural Networks through Reproducing Kernel Banach Spaces](https://mlsanity.com/paper/2211.05020/) (2022)
- [A Unified Approach to Controlling Implicit Regularization via Mirror Descent](https://mlsanity.com/paper/2306.13853/) (2023)
- [Sparse Representer Theorems for Learning in Reproducing Kernel Banach Spaces](https://mlsanity.com/paper/2305.12584/) (2023)
- [Random Fourier Features for Asymmetric Kernels](https://mlsanity.com/paper/2209.08461/) (2022)
- [Function-Space Optimality of Neural Architectures with Multivariate Nonlinearities](https://mlsanity.com/paper/2310.03696/) (2023)
- [A simple proof of the Baillon-Haddad theorem on open subsets of Hilbert spaces](https://mlsanity.com/paper/2204.00282/) (2022)
- [Learning Analysis of Kernel Ridgeless Regression with Asymmetric Kernel Learning](https://mlsanity.com/paper/2406.01435/) (2024)

## Built on by (most cited first, 2 of 2)

- [Solving Nonlinear PDEs with Sparse Radial Basis Function Networks](https://mlsanity.com/paper/2505.07765/) (2025)
- [A Gap Between Decision Trees and Neural Networks](https://mlsanity.com/paper/2601.03919/) (2026)

## Abstract (arXiv)

Recent advances in machine learning have led to increased interest in reproducing kernel Banach spaces (RKBS) as a more general framework that extends beyond reproducing kernel Hilbert spaces (RKHS). These works have resulted in the formulation of representer theorems under several regularized learning schemes. However, little is known about an optimization method that encompasses these results in this setting. This paper addresses a learning problem on Banach spaces endowed with a reproducing kernel, focusing on efficient optimization within RKBS. To tackle this challenge, we propose an algorithm based on mirror descent (MDA). Our approach involves an iterative method that employs gradient steps in the dual space of the Banach space using the reproducing kernel. We analyze the convergence properties of our algorithm under various assumptions and establish two types of results: first, we identify conditions under which a linear convergence rate is achievable, akin to optimization in the Euclidean setting, and provide a proof of the linear rate; second, we demonstrate a standard convergence rate in a constrained setting. Moreover, to instantiate this algorithm in practice, we introduce a novel family of RKBSs with p-norm (p neq 2), characterized by both an explicit dual map and a kernel.