mlsanityData to 14 Oct 2026

A pullback-corrected scalar auxiliary variable optimizer with momentum and adaptive mobility

arXiv 2609.13569 ↗First posted 11 Sep 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Objectives in scientific machine learning are often prescribed as a sum of several terms, such as the residual, boundary, initial, and data losses of a physics-informed neural network. In the pullback-corrected scalar auxiliary variable (PB--SAV) method, one scalar tracks the shifted objective while the component gradients build a positive semidefinite curvature correction of rank at most the number of components. We carry that correction into an optimizer with momentum and an adaptive mobility, applying it to the gradient and the stored momentum in a single implicit solve. A mobility that is nonincreasing in the Loewner order yields an exact modified energy law, covering Euclidean and AMSGrad-type choices; the corresponding identity for momentum appended after the solve carries a cross term of indefinite sign. For a fixed mobility we give a necessary and sufficient condition for local stability at a stationary point, depending on the Hessian minus twice the correction, and show that it also gives local geometric convergence for every scalar relaxation sequence. The implicit solve reduces to a dense system whose order is the number of components. In the forward Burgers comparison, four components reduce the mean tail objective by 64.7% and the final solution error by 50.2% relative to one component at the same learning rate and momentum settings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 31 references that are in the map. First posted on arXiv 11 Sep 2026.

Builds on

6 of 31 references.

References · 6

  1. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  2. 2019On the Convergence of Adam and Beyondcited by 841
  3. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 634
  4. 2026Muon with Spectral Guidance: Efficient Optimization for Scientific Machine Learningcited by 4
  5. 2022An Adaptive Gradient Method with Energy and Momentumcited by 3
  6. 2022Improving Levenberg-Marquardt Algorithm for Neural Networkscited by 2

Built on by

Not cited yet.