# Towards General Neural Surrogate Solvers with Specialized Neural Accelerators

arXiv: https://arxiv.org/abs/2405.02351 · HTML: https://mlsanity.com/paper/2405.02351/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 54 references that are in the map.
- First posted on arXiv 2 May 2024.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Empirical Evaluation of Rectified Activations in Convolutional Network](https://mlsanity.com/paper/1505.00853/) (2015)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [Factorized Fourier Neural Operators](https://mlsanity.com/paper/2111.13802/) (2021)
- [Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers](https://mlsanity.com/paper/2111.13587/) (2021)
- [PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network](https://mlsanity.com/paper/1812.04426/) (2018)
- [Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10566/) (2017)
- [Domain Agnostic Fourier Neural Operators](https://mlsanity.com/paper/2305.00478/) (2023)
- [NeurOLight: A Physics-Agnostic Neural Operator Enabling Parametric Photonic Device Simulation](https://mlsanity.com/paper/2209.10098/) (2022)
- [NUNO: A General Framework for Learning Parametric PDEs with Non-Uniform Data](https://mlsanity.com/paper/2305.18694/) (2023)
- [WaveY-Net: Physics-augmented deep learning for high-speed electromagnetic simulation and optimization](https://mlsanity.com/paper/2203.01248/) (2022)
- [Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networks](https://mlsanity.com/paper/2206.14862/) (2022)

## Built on by (most cited first, 9 of 9)

- [A multi-agentic framework for real-time, autonomous freeform metasurface design](https://mlsanity.com/paper/2503.20479/) (2025)
- [Operator Learning with Domain Decomposition for Geometry Generalization in PDE Solving](https://mlsanity.com/paper/2504.00510/) (2025)
- [Accurate and scalable deep Maxwell solvers using multilevel iterative methods](https://mlsanity.com/paper/2509.03622/) (2025)
- [GeoMaNO: Geometric Mamba Neural Operator for Partial Differential Equations](https://mlsanity.com/paper/2505.12020/) (2025)
- [Generalizing Deep Surrogate Solvers for Broadband Electromagnetic Field Prediction at Unseen Wavelengths](https://mlsanity.com/paper/2408.02971/) (2024)
- [A Differentiable Ray-Wave Framework for Hybrid Refractive-Diffractive System Modeling and Optimization](https://mlsanity.com/paper/2605.15418/) (2026)
- [Inductively Scalable, Single-Step Neural Surrogates for Wave-Scattering Inverse Problems](https://mlsanity.com/paper/2608.17344/) (2026)
- [Learnable composition for neural operators](https://mlsanity.com/paper/2609.03069/) (2026)
- [A Composable AI-Accelerated Iterative Solver for 3D-IC Thermal Modeling](https://mlsanity.com/paper/2610.02461/) (2026)

## Abstract (arXiv)

Surrogate neural network-based partial differential equation (PDE) solvers have the potential to solve PDEs in an accelerated manner, but they are largely limited to systems featuring fixed domain sizes, geometric layouts, and boundary conditions. We propose Specialized Neural Accelerator-Powered Domain Decomposition Methods (SNAP-DDM), a DDM-based approach to PDE solving in which subdomain problems containing arbitrary boundary conditions and geometric parameters are accurately solved using an ensemble of specialized neural operators. We tailor SNAP-DDM to 2D electromagnetics and fluidic flow problems and show how innovations in network architecture and loss function engineering can produce specialized surrogate subdomain solvers with near unity accuracy. We utilize these solvers with standard DDM algorithms to accurately solve freeform electromagnetics and fluids problems featuring a wide range of domain sizes.