mlsanityData to 7 Oct 2026

Duality for Neural Networks through Reproducing Kernel Banach Spaces

arXiv 2211.05020 ↗First posted 9 Nov 2022math.FA cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Reproducing Kernel Hilbert spaces (RKHS) have been a very successful tool in various areas of machine learning. Recently, Barron spaces have been used to prove bounds on the generalisation error for neural networks. Unfortunately, Barron spaces cannot be understood in terms of RKHS due to the strong nonlinear coupling of the weights. This can be solved by using the more general Reproducing Kernel Banach spaces (RKBS). We show that these Barron spaces belong to a class of integral RKBS. This class can also be understood as an infinite union of RKHS spaces. Furthermore, we show that the dual space of such RKBSs, is again an RKBS where the roles of the data and parameters are interchanged, forming an adjoint pair of RKBSs including a reproducing kernel. This allows us to construct the saddle point problem for neural networks, which can be used in the whole field of primal-dual optimisation.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 16 of its 47 references that are in the map. First posted on arXiv 9 Nov 2022.

Builds on

16 of 47 references.

References · 16

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  2. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
  3. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 267
  4. 2015On the Equivalence between Kernel Quadrature Rules and Random Feature Expansionscited by 136
  5. 2020Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don'tcited by 94
  6. 2018A Priori Estimates of the Population Risk for Two-layer Neural Networkscited by 76
  7. 2020Representation formulas and pointwise properties for Barron functionscited by 62
  8. 2021Understanding neural networks with reproducing kernel Banach spacescited by 50
  9. 2019The Barron Space and the Flow-induced Function Spaces for Neural Network Modelscited by 39
  10. 2020On the Banach spaces associated with multi-layer ReLU networks: Function representation, approximation theory and gradient descent dynamicscited by 39
  11. 2018A representer theorem for deep neural networkscited by 38
  12. 2019On Reproducing Kernel Banach Spaces: Generic Definitions and Unified Framework of Constructionscited by 29
4 more
  1. 2018Sparsity of solutions for variational inverse problems with finite-dimensional datacited by 25
  2. 2020Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernelscited by 19
  3. 2020Explicit Regularization of Stochastic Gradient Methods through Dualitycited by 6
  4. 2021Neural Architecture Search via Bregman Iterationscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 6; 2025: 1; 2026: 2.

15 papers.

Sort:

  1. 2023Sampling weights of deep neural networkscited by 20
  2. 2023Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compressioncited by 17
  3. 2023Sparse Representer Theorems for Learning in Reproducing Kernel Banach Spacescited by 10
  4. 2024Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networkscited by 8
  5. 2023Function-Space Optimality of Neural Architectures with Multivariate Nonlinearitiescited by 5
  6. 2023Embeddings between Barron spaces with higher order activation functionscited by 4
  7. 2024Hypothesis Spaces for Deep Learningcited by 4
  8. 2024A Lipschitz spaces view of infinitely wide shallow neural networkscited by 2
  9. 2024Mirror Descent on Reproducing Kernel Banach Spacescited by 2
  10. 2025Vector-Valued Reproducing Kernel Banach Spaces for Neural Networks and Operatorscited by 2
  11. 2023Learning a Sparse Representation of Barron Functions with the Inverse Scale Space Flowcited by 1
  12. 2024Novel Kernel Models and Exact Representor Theory for Neural Networks Beyond the Over-Parameterized Regimecited by 0
3 more
  1. 2024Decomposition of one-layer neural networks via the infinite sum of reproducing kernel Banach spacescited by 0
  2. 2026A Dual Certificate Approach to Sparsity in Infinite-Width Shallow Neural Networkscited by 0
  3. 2026Exact Convex Reformulations of Linear Neural Networks via Completely Positive Liftingcited by 0