mlsanityData to 14 Oct 2026

Two-layer neural networks with values in a Banach space

arXiv 2105.02095 ↗First posted 5 May 2021cs.LG cs.NA math.FAcited by 26

Not on the map. Why?

Abstract (arXiv)

We study two-layer neural networks whose domain and range are Banach spaces with separable preduals. In addition, we assume that the image space is equipped with a partial order, i.e. it is a Riesz space. As the nonlinearity we choose the lattice operation of taking the positive part; in case of mathbb Rd-valued neural networks this corresponds to the ReLU activation function. We prove inverse and direct approximation theorems with Monte-Carlo rates for a certain class of functions, extending existing results for the finite-dimensional case. In the second part of the paper, we study, from the regularisation theory viewpoint, the problem of finding optimal representations of such functions via signed measures on a latent space from a finite number of noisy observations. We discuss regularity conditions known as source conditions and obtain convergence rates in a Bregman distance for the representing measure in the regime when both the noise level goes to zero and the number of samples goes to infinity at appropriate rates.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 23 of its 69 references that are in the map. First posted on arXiv 5 May 2021.

Builds on

23 of 69 references.

References · 23

  1. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  2. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  3. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 734
  4. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 655
  5. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 466
  6. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 426
  7. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 331
  8. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 268
  9. 2020Model Reduction and Neural Networks for Parametric PDEscited by 258
  10. 2020The Random Feature Model for Input-Output Maps between Banach Spacescited by 90
  11. 2017A Review of Convolutional Neural Networks for Inverse Problems in Imagingcited by 81
  12. 2019A Theoretical Analysis of Deep Neural Networks and Parametric PDEscited by 74
11 more
  1. 2016Optimal Rates For Regularization Of Statistical Inverse Learning Problemscited by 68
  2. 2020Representation formulas and pointwise properties for Barron functionscited by 62
  3. 2019The Barron Space and the Flow-induced Function Spaces for Neural Network Modelscited by 39
  4. 2018Sparsity of solutions for variational inverse problems with finite-dimensional datacited by 25
  5. 2018On Representer Theorems and Convex Regularizationcited by 24
  6. 2019A unifying representer theorem for inverse problems and machine learningcited by 24
  7. 2013Exact Support Recovery for Sparse Spikes Deconvolutioncited by 22
  8. 2020On Sparsity in Overparametrised Shallow ReLU Networkscited by 9
  9. 2021On Energy-Based Models with Overparametrized Shallow Neural Networkscited by 8
  10. 2021Convex regularization in statistical inverse learning problemscited by 7
  11. 2020Variational regularisation for inverse problems with imperfect forward operators and general noise modelscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 2; 2023: 4; 2024: 9; 2025: 5; 2026: 4.

26 papers.

Sort:

  1. 2024Operator Learning: Algorithms and Analysiscited by 85
  2. 2021Convergence Rates for Learning Linear Operators from Noisy Datacited by 55
  3. 2024Data Complexity Estimates for Operator Learningcited by 26
  4. 2022Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysiscited by 22
  5. 2024Operator Learning Using Random Features: A Tool for Scientific Computingcited by 21
  6. 2023Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compressioncited by 18
  7. 2024Mixtures of Neural Operators Reduce Active Complexity in Operator Learningcited by 16
  8. 2024Operator Learning of Lipschitz Operators: An Information-Theoretic Perspectivecited by 11
  9. 2023Weighted variation spaces and approximation by shallow ReLU networkscited by 7
  10. 2025Generative Neural Operators of Log-Complexity Can Simultaneously Solve Infinitely Many Convex Programscited by 7
  11. 2023A convergence result of a continuous model of deep learning via a Lojasiewicz--Simon inequalitycited by 5
  12. 2023Inverse Problems with Learned Forward Operatorscited by 4
14 more
  1. 2024Universal approximation theorem for neural networks with inputs from a topological vector spacecited by 4
  2. 2025New universal operator approximation theorem for encoder-decoder architecturescited by 4
  3. 2025One model to solve them all: 2BSDE families via neural operatorscited by 4
  4. 2022A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networkscited by 3
  5. 2021Integral representations of shallow neural network with Rectified Power Unit activation functioncited by 2
  6. 2024A Lipschitz spaces view of infinitely wide shallow neural networkscited by 2
  7. 2024Polynomial Scaling is Possible For Neural Operator Approximations of Structured Families of BSDEscited by 2
  8. 2025On shallow feedforward neural networks with inputs from a topological spacecited by 2
  9. 2025Vector-Valued Reproducing Kernel Banach Spaces for Neural Networks and Operatorscited by 2
  10. 2026Variation Spaces for Encoder--Decoder Neural Operators: Approximation and Generalizationcited by 1
  11. 2024The Effects of Multi-Task Learning on ReLU Neural Network Functionscited by 0
  12. 2026Extending Neural Operators: Robust Handling of Functions Beyond the Training Setcited by 0
  13. 2026Approximation of Maximally Monotone Operators : A Graph Convergence Perspectivecited by 0
  14. 2026Resolution-Consistent Greedy Neural Approximation on Infinite-Dimensional Spacescited by 0