# Blending Neural Operators and Relaxation Methods in PDE Numerical Solvers

arXiv: https://arxiv.org/abs/2208.13273 · HTML: https://mlsanity.com/paper/2208.13273/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 102 references that are in the map.
- First posted on arXiv 28 Aug 2022.

## Builds on (most cited first, 29 of 29)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers](https://mlsanity.com/paper/2007.00016/) (2020)
- [Accelerating Eulerian Fluid Simulation With Convolutional Networks](https://mlsanity.com/paper/1607.03597/) (2016)
- [A physics-informed variational DeepONet for predicting the crack path in brittle materials](https://mlsanity.com/paper/2108.06905/) (2021)
- [Learning Operators with Coupled Attention](https://mlsanity.com/paper/2201.01032/) (2022)
- [Deep transfer operator learning for partial differential equations under conditional shift](https://mlsanity.com/paper/2204.09810/) (2022)
- [Learning to Optimize Multigrid PDE Solvers](https://mlsanity.com/paper/1902.10248/) (2019)
- [Learning Neural PDE Solvers with Convergence Guarantees](https://mlsanity.com/paper/1906.01200/) (2019)
- [Learning two-phase microstructure evolution using neural operators and autoencoder architectures](https://mlsanity.com/paper/2204.07230/) (2022)
- [A physics-informed operator regression framework for extracting data-driven continuum models](https://mlsanity.com/paper/2009.11992/) (2020)
- [Learning Algebraic Multigrid Using Graph Neural Networks](https://mlsanity.com/paper/2003.05744/) (2020)
- [Interfacing Finite Elements with Deep Neural Operators for Fast Multiscale Modeling of Mechanics Problems](https://mlsanity.com/paper/2203.00003/) (2022)
- [MgNet: A Unified Framework of Multigrid and Convolutional Neural Network](https://mlsanity.com/paper/1901.10415/) (2019)
- [Physics-Informed Neural Networks for Nonhomogeneous Material Identification in Elasticity Imaging](https://mlsanity.com/paper/2009.04525/) (2020)
- [Simulating progressive intramural damage leading to aortic dissection using an operator-regression neural network](https://mlsanity.com/paper/2108.11985/) (2021)
- [MeshingNet: A New Mesh Generation Method based on Deep Learning](https://mlsanity.com/paper/2004.07016/) (2020)
- [Multigrid-augmented deep learning preconditioners for the Helmholtz equation](https://mlsanity.com/paper/2203.11025/) (2022)
- [Systems Biology: Identifiability analysis and parameter identification via systems-biology informed neural networks](https://mlsanity.com/paper/2202.01723/) (2022)
- [G2Phinet: Relating Genotype and Biomechanical Phenotype of Tissues with Deep Learning](https://mlsanity.com/paper/2208.09889/) (2022)
- [A unified analysis framework for iterative parallel-in-time algorithms](https://mlsanity.com/paper/2203.16069/) (2022)

## Built on by (most cited first, 30 of 54)

- [Mitigating spectral bias for the multiscale operator learning](https://mlsanity.com/paper/2210.10890/) (2022)
- [Mitigating Spectral Bias in Neural Operators via High-Frequency Scaling for Physical Systems](https://mlsanity.com/paper/2503.13695/) (2025)
- [Real-time Inference and Extrapolation via a Diffusion-inspired Temporal Transformer Operator (DiTTO)](https://mlsanity.com/paper/2307.09072/) (2023)
- [Accelerating PDE Data Generation via Differential Operator Action in Solution Space](https://mlsanity.com/paper/2402.05957/) (2024)
- [A hybrid iterative method based on MIONet for PDEs: Theory and numerical examples](https://mlsanity.com/paper/2402.07156/) (2024)
- [Accelerating Data Generation for Neural Operators via Krylov Subspace Recycling](https://mlsanity.com/paper/2401.09516/) (2024)
- [Multigrid-Augmented Deep Learning Preconditioners for the Helmholtz Equation using Compact Implicit Layers](https://mlsanity.com/paper/2306.17486/) (2023)
- [Physics-aware Machine Learning Revolutionizes Scientific Paradigm for Machine Learning and Process-based Hydrology](https://mlsanity.com/paper/2310.05227/) (2023)
- [Neural Green's Operators for Parametric Partial Differential Equations](https://mlsanity.com/paper/2406.01857/) (2024)
- [Binned Spectral Power Loss for Improved Prediction of Chaotic Systems](https://mlsanity.com/paper/2502.00472/) (2025)
- [Enabling Local Neural Operators to perform Equation-Free System-Level Analysis](https://mlsanity.com/paper/2505.02308/) (2025)
- [NOWS: Neural Operator Warm Starts for Accelerating Iterative Solvers](https://mlsanity.com/paper/2511.02481/) (2025)
- [A Generative Modeling Framework for Inferring Families of Biomechanical Constitutive Laws in Data-Sparse Regimes](https://mlsanity.com/paper/2305.03184/) (2023)
- [MyCrunchGPT: A chatGPT assisted framework for scientific machine learning](https://mlsanity.com/paper/2306.15551/) (2023)
- [Fast meta-solvers for 3D complex-shape scatterers using neural operators trained on a non-scattering problem](https://mlsanity.com/paper/2405.12380/) (2024)
- [A Pretraining-Finetuning Computational Framework for Material Homogenization](https://mlsanity.com/paper/2404.07943/) (2024)
- [Neural-Initialized Newton: Accelerating Nonlinear Finite Elements via Operator Learning](https://mlsanity.com/paper/2511.06802/) (2025)
- [Leveraging Operator Learning to Accelerate Convergence of the Preconditioned Conjugate Gradient Method](https://mlsanity.com/paper/2508.00101/) (2025)
- [Accurate and scalable deep Maxwell solvers using multilevel iterative methods](https://mlsanity.com/paper/2509.03622/) (2025)
- [A Neural-Operator Preconditioned Newton Method for Accelerated Nonlinear Solvers](https://mlsanity.com/paper/2511.08811/) (2025)
- [Out-of-Distribution Generalization for Neural Physics Solvers](https://mlsanity.com/paper/2601.19091/) (2026)
- [A fast neural hybrid Newton solver adapted to implicit methods for nonlinear dynamics](https://mlsanity.com/paper/2407.03945/) (2024)
- [Self-composing neural operators for high-frequency and multiscale PDE surrogates](https://mlsanity.com/paper/2508.20650/) (2025)
- [Learning Singularity-Encoded Green's Functions with Application to Iterative Methods](https://mlsanity.com/paper/2509.11580/) (2025)
- [Hybrid Iterative Solvers with Geometry-Aware Neural Preconditioners for Parametric PDEs](https://mlsanity.com/paper/2512.14596/) (2025)
- [HyCOP: Hybrid Composition Operators for Interpretable Learning of PDEs](https://mlsanity.com/paper/2605.00820/) (2026)
- [Towards Automated Algebraic Multigrid Preconditioner Design Using Genetic Programming for Large-Scale Laser Beam Welding Simulations](https://mlsanity.com/paper/2412.08186/) (2024)
- [Fredholm Neural Networks for inverse problems in elliptic PDEs](https://mlsanity.com/paper/2507.06038/) (2025)
- [Accelerating Conjugate Gradient Solvers for Homogenization Problems with Unitary Neural Operators](https://mlsanity.com/paper/2508.02681/) (2025)
- [Learning Sparse Approximate Inverse Preconditioners for Conjugate Gradient Solvers on GPUs](https://mlsanity.com/paper/2510.27517/) (2025)

## Abstract (arXiv)

Neural networks suffer from spectral bias having difficulty in representing the high frequency components of a function while relaxation methods can resolve high frequencies efficiently but stall at moderate to low frequencies. We exploit the weaknesses of the two approaches by combining them synergistically to develop a fast numerical solver of partial differential equations (PDEs) at scale. Specifically, we propose HINTS, a hybrid, iterative, numerical, and transferable solver by integrating a Deep Operator Network (DeepONet) with standard relaxation methods, leading to parallel efficiency and algorithmic scalability for a wide class of PDEs, not tractable with existing monolithic solvers. HINTS balances the convergence behavior across the spectrum of eigenmodes by utilizing the spectral bias of DeepONet, resulting in a uniform convergence rate and hence exceptional performance of the hybrid solver overall. Moreover, HINTS applies to large-scale, multidimensional systems, it is flexible with regards to discretizations, computational domain, and boundary conditions.