mlsanityData to 7 Oct 2026

Understanding neural networks with reproducing kernel Banach spaces

arXiv 2109.09710 ↗First posted 20 Sep 2021stat.ML cs.LG math.FAcited by 50

Not on the map. Why?

Abstract (arXiv)

Characterizing the function spaces corresponding to neural networks can provide a way to understand their properties. In this paper we discuss how the theory of reproducing kernel Banach spaces can be used to tackle this challenge. In particular, we prove a representer theorem for a wide class of reproducing kernel Banach spaces that admit a suitable integral representation and include one hidden layer neural networks of possibly infinite width. Further, we show that, for a suitable class of ReLU activation functions, the norm in the corresponding reproducing kernel Banach space can be characterized in terms of the inverse Radon transform of a bounded real measure, with norm given by the total variation norm of the measure. Our analysis simplifies and extends recent results in [34,29,30].

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 17 of its 69 references that are in the map. First posted on arXiv 20 Sep 2021.

Builds on

17 of 69 references.

References · 17

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  3. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  4. 2017Deep Neural Networks as Gaussian Processescited by 617
  5. 2016Kernel Mean Embedding of Distributions: A Review and Beyondcited by 360
  6. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
  7. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 267
  8. 2016Generalization Properties of Learning with Random Featurescited by 163
  9. 2015Path-SGD: Path-Normalized Optimization in Deep Neural Networkscited by 117
  10. 2019A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Casecited by 113
  11. 2019How do infinite width bounded norm networks look in function space?cited by 105
  12. 2019On the Power and Limitations of Random Features for Understanding Neural Networkscited by 103
5 more
  1. 2019Approximation spaces of deep neural networkscited by 51
  2. 2019On Reproducing Kernel Banach Spaces: Generic Definitions and Unified Framework of Constructionscited by 29
  3. 2018Sparsity of solutions for variational inverse problems with finite-dimensional datacited by 25
  4. 2018On Representer Theorems and Convex Regularizationcited by 24
  5. 2019A unifying representer theorem for inverse problems and machine learningcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 15; 2024: 13; 2025: 7; 2026: 8.

50 papers.

Sort:

  1. 2023Nonparametric regression using over-parameterized shallow ReLU neural networkscited by 24
  2. 2023Deep Learning Meets Sparse Regularization: A Signal Processing Perspectivecited by 21
  3. 2022Ridges, Neural Networks, and the Radon Transformcited by 18
  4. 2023Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compressioncited by 17
  5. 2022Duality for Neural Networks through Reproducing Kernel Banach Spacescited by 15
  6. 2023Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regressioncited by 15
  7. 2023Sparse Representer Theorems for Learning in Reproducing Kernel Banach Spacescited by 10
  8. 2024Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networkscited by 8
  9. 2022Stability of Image-Reconstruction Algorithmscited by 7
  10. 2022Random Fourier Features for Asymmetric Kernelscited by 7
  11. 2023ReLU Neural Networks with Linear Layers are Biased Towards Single- and Multi-Index Modelscited by 7
  12. 2024Dataset Condensation with Latent Quantile Matchingcited by 6
38 more
  1. 2023Neural Hilbert Ladders: Multi-Layer Neural Networks in Function Spacecited by 5
  2. 2023Function-Space Optimality of Neural Architectures with Multivariate Nonlinearitiescited by 5
  3. 2024ReLUs Are Sufficient for Learning Implicit Neural Representationscited by 5
  4. 2024Weighted Sobolev Approximation Rates for Neural Networks on Unbounded Domainscited by 5
  5. 2022From Kernel Methods to Neural Networks: A Unifying Variational Formulationcited by 4
  6. 2023Embeddings between Barron spaces with higher order activation functionscited by 4
  7. 2023Tractability of approximation by general shallow networkscited by 4
  8. 2024Hypothesis Spaces for Deep Learningcited by 4
  9. 2024Neural reproducing kernel Banach spaces and representer theorems for deep networkscited by 4
  10. 2021Tighter Sparse Approximation Bounds for ReLU Neural Networkscited by 3
  11. 2024Random ReLU Neural Networks as Non-Gaussian Processescited by 3
  12. 2025Solving Nonlinear PDEs with Sparse Radial Basis Function Networkscited by 3
  13. 2022The Mori-Zwanzig formulation of deep learningcited by 2
  14. 2024Nonuniform random feature models using derivative informationcited by 2
  15. 2024A Lipschitz spaces view of infinitely wide shallow neural networkscited by 2
  16. 2024Mirror Descent on Reproducing Kernel Banach Spacescited by 2
  17. 2025A Gap Between the Gaussian RKHS and Neural Networks: An Infinite-Center Asymptotic Analysiscited by 2
  18. 2025Manipulating Transformer-Based Models: Controllability, Steerability, and Robust Interventionscited by 2
  19. 2025Vector-Valued Reproducing Kernel Banach Spaces for Neural Networks and Operatorscited by 2
  20. 2023Gradient Descent in Neural Networks as Sequential Learning in RKBScited by 1
  21. 2023Approximation by non-symmetric networks for cross-domain learningcited by 1
  22. 2023Barron Space for Graph Convolution Neural Networkscited by 1
  23. 2023Learning a Sparse Representation of Barron Functions with the Inverse Scale Space Flowcited by 1
  24. 2023Outlier-robust neural network training: variation regularization meets trimmed loss to prevent functional breakdowncited by 0
  25. 2024Novel Kernel Models and Exact Representor Theory for Neural Networks Beyond the Over-Parameterized Regimecited by 0
  26. 2024Decomposition of one-layer neural networks via the infinite sum of reproducing kernel Banach spacescited by 0
  27. 2024The Effects of Multi-Task Learning on ReLU Neural Network Functionscited by 0
  28. 2025Irregular Sampling of High-Dimensional Functions in Reproducing Kernel Hilbert Spacescited by 0
  29. 2025Sobolev norm inconsistency of kernel interpolationcited by 0
  30. 2025DeepMartingale: Duality of the Optimal Stopping Problem with Expressivity and High-Dimensional Hedgingcited by 0
  31. 2026Sparse RBF Networks for PDEs and nonlocal equations: function space theory, operator calculus, and training algorithmscited by 0
  32. 2026Beyond NNGP: Large Deviations and Feature Learning in Bayesian Neural Networkscited by 0
  33. 2026Self-Regularized Learning Methodscited by 0
  34. 2026A Dual Certificate Approach to Sparsity in Infinite-Width Shallow Neural Networkscited by 0
  35. 2026Physics-Modeled Neural Networkscited by 0
  36. 2026Transformers Can Implement Preconditioned Richardson Iteration for In-Context Gaussian Kernel Regressioncited by 0
  37. 2026Convex Hybrid Modeling: An Operator-Based Approachcited by 0
  38. 2026Gaussian Processes with Sample Paths in Reproducing Kernel Banach Spacescited by 0