mlsanityData to 7 Oct 2026

Hybrid Least Squares/Gradient Descent Methods for MIONets

arXiv 2607.06976 ↗First posted 8 Jul 2026cs.LG cs.AI cs.NAcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose an efficient hybrid least squares/gradient descent (LSGD) method for MIONets to accelerate training. This method generalizes the LSGD method for DeepONets. Since MIONet is the sum of the entrywise product of multiple branch networks and a trunk network, it can be viewed as a multilinear function with respect to the last layer parameters of each branch network. These sets of parameters can be optimized using the alternating least squares method, where we solve the LS system for a single branch network in turn. To handle the large-sized system matrix, we introduce Kronecker and Khatri-Rao products and tensor permutation matrices to factor the large matrix into small ones. Our method is compatible with a general type of L2 loss with regularization terms for the last layer parameters of each branch, where linear operators can be applied to the MIONet output in each loss term.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 32 references that are in the map. First posted on arXiv 8 Jul 2026.

Builds on

17 of 32 references.

References · 17

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  3. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  4. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  5. 2015Automatic differentiation in machine learning: a surveycited by 846
  6. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 722
  7. 2020Model Reduction and Neural Networks for Parametric PDEscited by 257
  8. 2021Multiwavelet-based Operator Learning for Differential Equationscited by 229
  9. 2022MIONet: Learning multiple-input operators via tensor productcited by 180
  10. 2022Multifidelity deep neural operators for efficient learning of partial differential equations with application to fast inverse design of nanoscale heat transportcited by 92
  11. 2022NOMAD: Nonlinear Manifold Decoders for Operator Learningcited by 81
  12. 2024Geom-DeepONet: A Point-cloud-based Deep Operator Network for Field Predictions on 3D Parameterized Geometriescited by 47
5 more
  1. 2019Robust Training and Initialization of Deep Neural Networks: An Adaptive Basis Viewpointcited by 42
  2. 2023HyperDeepONet: learning operator with complex target function space using the limited resources via hypernetworkcited by 25
  3. 2024A hybrid iterative method based on MIONet for PDEs: Theory and numerical examplescited by 13
  4. 2025ELM-DeepONets: Backpropagation-Free Training of Deep Operator Networks via Extreme Learning Machinescited by 3
  5. 2025Hybrid Least Squares/Gradient Descent Methods for DeepONetscited by 2

Built on by

Not cited yet.