Hybrid Least Squares/Gradient Descent Methods for DeepONets
Not on the map. Why?
Abstract (arXiv)
We propose an efficient hybrid least squares/gradient descent method to accelerate DeepONet training. Since the output of DeepONet can be viewed as linear with respect to the last layer parameters of the branch network, these parameters can be optimized using a least squares (LS) solve, and the remaining hidden layer parameters are updated by means of gradient descent form. However, building the LS system for all possible combinations of branch and trunk inputs yields a prohibitively large linear problem that is infeasible to solve directly. To address this issue, our method decomposes the large LS system into two smaller, more manageable subproblems unicodex2014 one for the branch network and one for the trunk network unicodex2014 and solves them separately. This method is generalized to a broader type of L2 loss with a regularization term for the last layer parameters, including the case of unsupervised learning with physics-informed loss.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 19 of its 37 references that are in the map. First posted on arXiv 21 Aug 2025.
Builds on
19 of 37 references.
References · 19
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
- 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
- 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
- 2015Automatic differentiation in machine learning: a surveycited by 855
- 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 734
- 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 543
- 2021Learning the solution operator of parametric partial differential equations with physics-informed DeepOnetscited by 540
- 2017The Expressive Power of Neural Networks: A View from the Widthcited by 320
- 2019Sparse Networks from Scratch: Faster Training without Losing Performancecited by 213
- 2022MIONet: Learning multiple-input operators via tensor productcited by 181
- 2018Universality of Deep Convolutional Neural Networkscited by 176
7 more
- 2018ResNet with one-neuron hidden layers is a Universal Approximatorcited by 79
- 2019Robust Training and Initialization of Deep Neural Networks: An Adaptive Basis Viewpointcited by 42
- 2024A Survey on Universal Approximation Theoremscited by 15
- 2021Parareal Neural Networks Emulating a Parallel-in-time Algorithmcited by 4
- 2021Two-level Group Convolutioncited by 4
- 2025ELM-DeepONets: Backpropagation-Free Training of Deep Operator Networks via Extreme Learning Machinescited by 3
- 2016Accelerating Deep Learning with Shrinkage and Recallcited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.