# Convex Optimization in Sums of Banach Spaces

arXiv: https://arxiv.org/abs/2104.13127 · HTML: https://mlsanity.com/paper/2104.13127/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Apr 2021.

## Built on by (most cited first, 11 of 11)

- [What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory](https://mlsanity.com/paper/2105.03361/) (2021)
- [Ridges, Neural Networks, and the Radon Transform](https://mlsanity.com/paper/2203.02543/) (2022)
- [Sparse Representer Theorems for Learning in Reproducing Kernel Banach Spaces](https://mlsanity.com/paper/2305.12584/) (2023)
- [Stability of Image-Reconstruction Algorithms](https://mlsanity.com/paper/2206.07128/) (2022)
- [Function-Space Optimality of Neural Architectures with Multivariate Nonlinearities](https://mlsanity.com/paper/2310.03696/) (2023)
- [Measuring Complexity of Learning Schemes Using Hessian-Schatten Total Variation](https://mlsanity.com/paper/2112.06209/) (2021)
- [From Kernel Methods to Neural Networks: A Unifying Variational Formulation](https://mlsanity.com/paper/2206.14625/) (2022)
- [Sparsest Univariate Learning Models Under Lipschitz Constraint](https://mlsanity.com/paper/2112.13542/) (2021)
- [On the Convergence of Stochastic Gradient Descent for Linear Inverse Problems in Banach Spaces](https://mlsanity.com/paper/2302.05197/) (2023)
- [Decomposition of one-layer neural networks via the infinite sum of reproducing kernel Banach spaces](https://mlsanity.com/paper/2409.18132/) (2024)
- [Generalized Splines and Gaussian Processes](https://mlsanity.com/paper/2608.28446/) (2026)

## Abstract (arXiv)

We characterize the solution of a broad class of convex optimization problems that address the reconstruction of a function from a finite number of linear measurements. The underlying hypothesis is that the solution is decomposable as a finite sum of components, where each component belongs to its own prescribed Banach space; moreover, the problem is regularized by penalizing some composite norm of the solution. We establish general conditions for existence and derive the generic parametric representation of the solution components. These representations fall into three categories depending on the underlying regularization norm: (i) a linear expansion in terms of predefined "kernels" when the component space is a reproducing kernel Hilbert space (RKHS), (ii) a non-linear (duality) mapping of a linear combination of measurement functionals when the component Banach space is strictly convex, and, (iii) an adaptive expansion in terms of a small number of atoms within a larger dictionary when the component Banach space is not strictly convex. Our approach generalizes and unifies a number of multi-kernel (RKHS) and sparse-dictionary learning techniques for compressed sensing available in the literature. It also yields the natural extension of the classical spline-fitting techniques in (semi-)RKHS to the abstract Banach-space setting.