# On Representer Theorems and Convex Regularization

arXiv: https://arxiv.org/abs/1806.09810 · HTML: https://mlsanity.com/paper/1806.09810/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2018.

## Built on by (most cited first, 24 of 24)

- [Understanding neural networks with reproducing kernel Banach spaces](https://mlsanity.com/paper/2109.09710/) (2021)
- [Two-layer neural networks with values in a Banach space](https://mlsanity.com/paper/2105.02095/) (2021)
- [Learning sparse features can lead to overfitting in neural networks](https://mlsanity.com/paper/2206.12314/) (2022)
- [Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compression](https://mlsanity.com/paper/2305.16534/) (2023)
- [Sparse Representer Theorems for Learning in Reproducing Kernel Banach Spaces](https://mlsanity.com/paper/2305.12584/) (2023)
- [Stability of Image-Reconstruction Algorithms](https://mlsanity.com/paper/2206.07128/) (2022)
- [Weighted variation spaces and approximation by shallow ReLU networks](https://mlsanity.com/paper/2307.15772/) (2023)
- [Function-Space Optimality of Neural Architectures with Multivariate Nonlinearities](https://mlsanity.com/paper/2310.03696/) (2023)
- [From Kernel Methods to Neural Networks: A Unifying Variational Formulation](https://mlsanity.com/paper/2206.14625/) (2022)
- [Hypothesis Spaces for Deep Learning](https://mlsanity.com/paper/2403.03353/) (2024)
- [FastPart: Over-Parameterized Stochastic Gradient Descent for Sparse optimisation on Measures](https://mlsanity.com/paper/2312.05993/) (2023)
- [Solving Nonlinear PDEs with Sparse Radial Basis Function Networks](https://mlsanity.com/paper/2505.07765/) (2025)
- [Off-the-grid learning of mixtures from a continuous dictionary](https://mlsanity.com/paper/2207.00171/) (2022)
- [Efficient Algorithms for Regularized Nonnegative Scale-invariant Low-rank Approximation Models](https://mlsanity.com/paper/2403.18517/) (2024)
- [Nonuniform random feature models using derivative information](https://mlsanity.com/paper/2410.02132/) (2024)
- [A Lipschitz spaces view of infinitely wide shallow neural networks](https://mlsanity.com/paper/2410.14591/) (2024)
- [Representation and Regression Problems in Neural Networks: Relaxation, Generalization, and Numerics](https://mlsanity.com/paper/2412.01619/) (2024)
- [A Dual Certificate Approach to Sparsity in Infinite-Width Shallow Neural Networks](https://mlsanity.com/paper/2603.17785/) (2026)
- [Fenchel-Young Duality Gaps: Certified Early Stopping for Regularized Inverse Problems](https://mlsanity.com/paper/2609.17629/) (2026)
- [Analysis of Regularized Learning in Banach Spaces for Linear-functional Data](https://mlsanity.com/paper/2109.03159/) (2021)
- [Accelerating Non-Negative and Bounded-Variable Linear Regression Algorithms with Safe Screening](https://mlsanity.com/paper/2202.07258/) (2022)
- [Simultaneous off-the-grid learning of mixtures issued from a continuous dictionary](https://mlsanity.com/paper/2210.16311/) (2022)
- [Decomposition of one-layer neural networks via the infinite sum of reproducing kernel Banach spaces](https://mlsanity.com/paper/2409.18132/) (2024)
- [Effective regions and kernels in continuous sparse regularisation, with application to sketched mixtures](https://mlsanity.com/paper/2507.08444/) (2025)

## Abstract (arXiv)

We establish a general principle which states that regularizing an inverse problem with a convex function yields solutions which are convex combinations of a small number of atoms. These atoms are identified with the extreme points and elements of the extreme rays of the regularizer level sets. An extension to a broader class of quasi-convex regularizers is also discussed. As a side result, we characterize the minimizers of the total gradient variation, which was still an unresolved problem.