mlsanityData to 7 Oct 2026

Hypothesis Spaces for Deep Learning

arXiv 2403.03353 ↗First posted 5 Mar 2024stat.ML cs.LG math.FAcited by 4

Not on the map. Why?

Abstract (arXiv)

This paper introduces a hypothesis space for deep learning based on deep neural networks (DNNs). By treating a DNN as a function of two variables - the input variable and the parameter variable - we consider the set of DNNs where the parameter variable belongs to a space of weight matrices and biases determined by a prescribed depth and layer widths. To construct a Banach space of functions of the input variable, we take the weak* closure of the linear span of this DNN set. We prove that the resulting Banach space is a reproducing kernel Banach space (RKBS) and explicitly construct its reproducing kernel. Furthermore, we investigate two learning models - regularized learning and the minimum norm interpolation (MNI) problem - within the RKBS framework by establishing representer theorems. These theorems reveal that the solutions to these learning problems can be expressed as a finite sum of kernel expansions based on training data.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 22 of its 47 references that are in the map. First posted on arXiv 5 Mar 2024.

Builds on

22 of 47 references.

References · 22

  1. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
  2. 2018Universality of Deep Convolutional Neural Networkscited by 176
  3. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 95
  4. 2016Deep vs. shallow networks : An approximation theory perspectivecited by 63
  5. 2021What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theorycited by 60
  6. 2021Understanding neural networks with reproducing kernel Banach spacescited by 50
  7. 2021Characterization of the Variation Spaces Corresponding to Shallow Neural Networkscited by 48
  8. 2019Nonlinear Approximation and (Deep) ReLU Networkscited by 45
  9. 2018A representer theorem for deep neural networkscited by 38
  10. 2021An error analysis of generative adversarial networks for learning distributionscited by 35
  11. 2017A representer theorem for deep kernel learningcited by 26
  12. 2018Sparsity of solutions for variational inverse problems with finite-dimensional datacited by 25
10 more
  1. 2018On Representer Theorems and Convex Regularizationcited by 24
  2. 2023Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compressioncited by 17
  3. 2022Duality for Neural Networks through Reproducing Kernel Banach Spacescited by 15
  4. 2021Convergence of Deep ReLU Networkscited by 14
  5. 2021Convergence of Deep Convolutional Neural Networkscited by 13
  6. 2021On the existence of global minima and convergence analyses for gradient descent methods in the training of deep neural networkscited by 12
  7. 2023Multi-Grade Deep Learningcited by 12
  8. 2023Sparse Representer Theorems for Learning in Reproducing Kernel Banach Spacescited by 10
  9. 2023Uniform Convergence of Deep Neural Networks with Lipschitz Continuous Activation Functions and Variable Widthscited by 5
  10. 2023Barron Space for Graph Convolution Neural Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Vector-Valued Reproducing Kernel Banach Spaces for Neural Networks and Operatorscited by 2
  2. 2026Multigrade Neural Network Approximationcited by 1
  3. 2026Featured Reproducing Kernel Banach Spaces for Learning and Neural Networkscited by 0
  4. 2026Transformers Can Implement Preconditioned Richardson Iteration for In-Context Gaussian Kernel Regressioncited by 0