# Spectrally Safe Neural Operator Warm-Starts for Large-Scale Newton Solvers

arXiv: https://arxiv.org/abs/2606.21828 · HTML: https://mlsanity.com/paper/2606.21828/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 46 references that are in the map.
- First posted on arXiv 20 Jun 2026.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Transolver: A Fast Transformer Solver for PDEs on General Geometries](https://mlsanity.com/paper/2402.02366/) (2024)
- [Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere](https://mlsanity.com/paper/2306.03838/) (2023)
- [Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries](https://mlsanity.com/paper/2502.02414/) (2025)
- [Multifidelity deep neural operators for efficient learning of partial differential equations with application to fast inverse design of nanoscale heat transport](https://mlsanity.com/paper/2204.06684/) (2022)
- [Blending Neural Operators and Relaxation Methods in PDE Numerical Solvers](https://mlsanity.com/paper/2208.13273/) (2022)
- [On the training and generalization of deep operator networks](https://mlsanity.com/paper/2309.01020/) (2023)
- [Elliptic PDE learning is provably data-efficient](https://mlsanity.com/paper/2302.12888/) (2023)
- [Finite Element Operator Network for Solving Elliptic-type parametric PDEs](https://mlsanity.com/paper/2308.04690/) (2023)
- [Spectral operator learning for parametric PDEs without data reliance](https://mlsanity.com/paper/2310.02013/) (2023)
- [NOWS: Neural Operator Warm Starts for Accelerating Iterative Solvers](https://mlsanity.com/paper/2511.02481/) (2025)
- [A Pretraining-Finetuning Computational Framework for Material Homogenization](https://mlsanity.com/paper/2404.07943/) (2024)
- [Neural-Initialized Newton: Accelerating Nonlinear Finite Elements via Operator Learning](https://mlsanity.com/paper/2511.06802/) (2025)
- [Operator Learning at Machine Precision](https://mlsanity.com/paper/2511.19980/) (2025)
- [A Neural-Operator Preconditioned Newton Method for Accelerated Nonlinear Solvers](https://mlsanity.com/paper/2511.08811/) (2025)
- [Error analysis for finite element operator learning methods for solving parametric second-order elliptic PDEs](https://mlsanity.com/paper/2404.17868/) (2024)

## Built on by (most cited first, 1 of 1)

- [Transferability of Learned States in Neural PDE Solvers](https://mlsanity.com/paper/2610.10972/) (2026)

## Abstract (arXiv)

Neural operators are increasingly used to warm-start Newton solvers for nonlinear PDEs, on the premise that a low test error places the initial guess inside the basin of attraction. We show that this premise is unreliable. An operator trained to the relative (L2) error (O(10-3)) can still produce an initial state in which the discrete Jacobian is indefinite, because the mean-squared training controls error on average while leaving localized pointwise violations of the underlying physics. For a nearly incompressible hyperelasticity problem, we trace this to the predicted volume change: the operator disperses (det F) well away from one, and the resulting Jacobian acquires negative eigenvalues even when the predicted field is visually indistinguishable from the reference. At a small scale, this is a nuisance; at a multi-million degree-of-freedom scale, it is disqualifying, since the conjugate gradient and other Krylov solvers needed for memory-feasible Newton steps assume a definite spectrum. We then show that a short, label-free fine-tuning phase -- penalizing the operator against the discrete energy, with no additional solution data -- shifts the Jacobian spectrum back to positive definite. Combined with an inexact outer loop, this gives a warm-started Newton method that converges across the full loading range where the unregularized operator fails, reaching up to 5.4(×) wall-clock speedup over incremental continuation on a 3D problem with 6.4 million degrees of freedom.