mlsanityData to 14 Oct 2026

Variation Spaces for Encoder--Decoder Neural Operators: Approximation and Generalization

arXiv 2606.01244 ↗First posted 31 May 2026stat.ML cs.LG cs.NAcited by 1

Not on the map. Why?

Abstract (arXiv)

Inspired by the function-space theory of neural networks, we formulate and analyze a variation space for nonlinear operators between Hilbert spaces, defined through vector-valued Borel measures of bounded variation. We characterize its unit ball as the closed convex hull of a vector-valued single-neuron dictionary in Bochner spaces. For the ReLU activation, the bounded linear operators in this space are precisely the Schatten-1 operators, with equivalent norms. For operators in this space, we establish encoder--decoder approximation bounds in the Bochner Lq-norm, where the error decomposes into input and output encoding errors and a finite-width term of order N-1/2. Under sub-Gaussian assumptions on the input and noise, we further derive high-probability generalization bounds for empirical least squares over path-norm-constrained encoder--decoder networks; the finite-sample contribution to the squared prediction error is of order K-1/2 up to logarithmic factors. The finite-width and finite-sample constants are independent of the encoding dimensions and bases, with the latter also independent of the network width. When the encoding errors decay algebraically, these bounds yield algebraic approximation and learning rates, in contrast to the complexity barriers for Lipschitz and Fr'echet differentiable operator classes.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 29 of its 52 references that are in the map. First posted on arXiv 31 May 2026.

Builds on

29 of 52 references.

References · 29

  1. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  2. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  3. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 734
  4. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 331
  5. 2020Model Reduction and Neural Networks for Parametric PDEscited by 258
  6. 2021Multiwavelet-based Operator Learning for Differential Equationscited by 234
  7. 2016A vector-contraction inequality for Rademacher complexitiescited by 142
  8. 2019A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Casecited by 114
  9. 2021Long-time integration of parametric evolution equations with physics-informed DeepONetscited by 89
  10. 2021Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networkscited by 86
  11. 2018A Priori Estimates of the Population Risk for Two-layer Neural Networkscited by 76
  12. 2023Neural Inverse Operators for Solving PDE Inverse Problemscited by 65
17 more
  1. 2022Deep Nonparametric Estimation of Operators between Infinite Dimensional Spacescited by 45
  2. 2023Operator learning with PCA-Net: upper and lower complexity boundscited by 44
  3. 2021Solving PDE-constrained Control Problems Using Operator Learningcited by 42
  4. 2021Near-Minimax Optimal Estimation With Shallow ReLU Neural Networkscited by 41
  5. 2023The Parametric Complexity of Operator Learningcited by 28
  6. 2021Two-layer neural networks with values in a Banach spacecited by 26
  7. 2024Data Complexity Estimates for Operator Learningcited by 26
  8. 2023Nonparametric regression using over-parameterized shallow ReLU neural networkscited by 24
  9. 2023Deep Operator Network Approximation Rates for Lipschitz Operatorscited by 24
  10. 2023Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compressioncited by 18
  11. 2024Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Studycited by 14
  12. 2024The Sample Complexity of Learning Lipschitz Operators with respect to Gaussian Measurescited by 11
  13. 2023Weighted variation spaces and approximation by shallow ReLU networkscited by 7
  14. 2025Towards Sharp Minimax Risk Bounds for Operator Learningcited by 5
  15. 2024Stochastic Gradient Descent for Operator Learning in Hilbert Spaces: Convergence Rates and Minimax Lower Boundscited by 2
  16. 2025Learning Operators by Regularized Stochastic Gradient Descent with Operator-valued Kernelscited by 2
  17. 2025A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Generalization Guarantees for Multi-Input Neural Operator Learning in Sobolev Spacescited by 3