mlsanityData to 7 Oct 2026

Orthogonal greedy algorithm for linear operator learning with shallow neural network

arXiv 2501.02791 ↗First posted 6 Jan 2025math.NA cs.LG cs.NAcited by 3

Not on the map. Why?

Abstract (arXiv)

Greedy algorithms, particularly the orthogonal greedy algorithm (OGA), have proven effective in training shallow neural networks for fitting functions and solving partial differential equations (PDEs). In this paper, we extend the application of OGA to the tasks of linear operator learning, which is equivalent to learning the kernel function through integral transforms. Firstly, a novel greedy algorithm is developed for kernel estimation rate in a new semi-inner product, which can be utilized to approximate the Green's function of linear PDEs from data. Secondly, we introduce the OGA for point-wise kernel estimation to further improve the approximation rate, achieving orders of accuracy improvement across various tasks and baseline models. In addition, we provide a theoretical analysis on the kernel estimation problem and the optimal approximation rates for both algorithms, establishing their efficacy and potential for future applications in PDEs and operator learning tasks.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 45 of its 85 references that are in the map. First posted on arXiv 6 Jan 2025.

Builds on

45 of 85 references.

References · 45

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  3. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  4. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 722
  5. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
  6. 2019DeepXDE: A deep learning library for solving differential equationscited by 617
  7. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 581
  8. 2021Learning the solution operator of parametric partial differential equations with physics-informed DeepOnetscited by 537
  9. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 416
  10. 2020Multipole Graph Neural Operator for Parametric Partial Differential Equationscited by 343
  11. 2021Choose a Transformer: Fourier or Galerkincited by 333
  12. 2023GNOT: A General Neural Operator Transformer for Operator Learningcited by 321
33 more
  1. 2021Factorized Fourier Neural Operatorscited by 237
  2. 2022U-NO: U-shaped Neural Operatorscited by 219
  3. 2022MIONet: Learning multiple-input operators via tensor productcited by 180
  4. 2023Scalable Transformer for PDE Surrogate Modelingcited by 123
  5. 2022Learning Operators with Coupled Attentioncited by 88
  6. 2021Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networkscited by 86
  7. 2021Improved architectures and training algorithms for deep operator networkscited by 85
  8. 2016Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with ℓ1 and ℓ0 Controlscited by 84
  9. 2020Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equationscited by 78
  10. 2020Rational neural networkscited by 75
  11. 2023A Mathematical Guide to Operator Learningcited by 74
  12. 2023Can Physics-Informed Neural Networks beat the Finite Element Method?cited by 68
  13. 2022Learning Deep Implicit Fourier Neural Operators (IFNOs) with Applications to Heterogeneous Material Modelingcited by 65
  14. 2020DeepGreen: Deep Learning of Green's Functions for Nonlinear Boundary Value Problemscited by 61
  15. 2019A Comparative Analysis of the Optimization and Generalization Property of Two-layer Neural Network and Random Feature Models Under Gradient Descent Dynamicscited by 49
  16. 2021Data-driven discovery of Green's functions with human-understandable deep learningcited by 43
  17. 2019MgNet: A Unified Framework of Multigrid and Convolutional Neural Networkcited by 41
  18. 2023MgNO: Efficient Parameterization of Linear Operators via Multigridcited by 40
  19. 2023Elliptic PDE learning is provably data-efficientcited by 26
  20. 2022On the Activation Function Dependence of the Spectral Bias of Neural Networkscited by 25
  21. 2021MOD-Net: A Machine Learning Approach via Model-Operator-Data Network for Solving PDEscited by 24
  22. 2022Transformer Meets Boundary Value Inverse Problemscited by 22
  23. 2023IB-UQ: Information bottleneck based uncertainty quantification for neural function regression and neural operator learningcited by 18
  24. 2021Learning Green's Functions of Linear Reaction-Diffusion Equations with Application to Fast Numerical Solvercited by 15
  25. 2022BI-GreenNet: Learning Green's functions by boundary integral networkcited by 13
  26. 2023TransNet: Transferable Neural Networks for Partial Differential Equationscited by 13
  27. 2023Deep Generalized Green's Functionscited by 5
  28. 2023An Enhanced V-cycle MgNet Model for Operator Learning in Numerical Partial Differential Equationscited by 4
  29. 2024Approximation with Random Shallow ReLU Networks with Applications to Model Reference Adaptive Controlcited by 4
  30. 2024A Nonoverlapping Domain Decomposition Method for Extreme Learning Machines: Elliptic Problemscited by 4
  31. 2021Tighter Sparse Approximation Bounds for ReLU Neural Networkscited by 3
  32. 2024Green Multigrid Networkcited by 3
  33. 2023FV-MgNet: Fully Connected V-cycle MgNet for Interpretable Time Series Forecastingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025A Deep Learning Framework for Multi-Operator Learning: Architectures and Approximation Theorycited by 6
  2. 2025MLPs and KANs for data-driven learning in physical problems: A performance comparisoncited by 1
  3. 2026Resolution-Consistent Greedy Neural Approximation on Infinite-Dimensional Spacescited by 0