# Physics-enhanced deep surrogates for partial differential equations

arXiv: https://arxiv.org/abs/2111.05841 · HTML: https://mlsanity.com/paper/2111.05841/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 75 references that are in the map.
- First posted on arXiv 10 Nov 2021.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Universal Differential Equations for Scientific Machine Learning](https://mlsanity.com/paper/2001.04385/) (2020)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Kernel and Rich Regimes in Overparametrized Models](https://mlsanity.com/paper/2002.09277/) (2020)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [Universal Approximation with Deep Narrow Networks](https://mlsanity.com/paper/1905.08539/) (2019)
- [Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems](https://mlsanity.com/paper/2003.04919/) (2020)
- [Don't Unroll Adjoint: Differentiating SSA-Form Programs](https://mlsanity.com/paper/1810.07951/) (2018)
- [A Framework for Machine Learning of Model Error in Dynamical Systems](https://mlsanity.com/paper/2107.06658/) (2021)
- [Deep Learning for Reduced Order Modelling and Efficient Temporal Evolution of Fluid Simulations](https://mlsanity.com/paper/2107.04556/) (2021)
- [A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients](https://mlsanity.com/paper/1809.07321/) (2018)
- [Active learning of deep surrogates for PDEs: Application to metasurface design](https://mlsanity.com/paper/2008.12649/) (2020)
- [Multi-fidelity Generative Deep Learning Turbulent Flows](https://mlsanity.com/paper/2006.04731/) (2020)
- [A three layer neural network can represent any multivariate function](https://mlsanity.com/paper/2012.03016/) (2020)

## Built on by (most cited first, 18 of 18)

- [Clifford Neural Layers for PDE Modeling](https://mlsanity.com/paper/2209.04934/) (2022)
- [Learning continuous models for continuous physics](https://mlsanity.com/paper/2202.08494/) (2022)
- [A neural operator-based surrogate solver for free-form electromagnetic inverse design](https://mlsanity.com/paper/2302.01934/) (2023)
- [Diffeomorphism Neural Operator for various domains and parameters of partial differential equations](https://mlsanity.com/paper/2402.12475/) (2024)
- [A Gaussian Process Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations](https://mlsanity.com/paper/2401.03492/) (2024)
- [Inverse Design in Nanophotonics via Representation Learning](https://mlsanity.com/paper/2507.00546/) (2025)
- [HiLAB: A Hybrid Inverse-Design Framework](https://mlsanity.com/paper/2505.17491/) (2025)
- [PAPM: A Physics-aware Proxy Model for Process Systems](https://mlsanity.com/paper/2407.05232/) (2024)
- [Building Trustworthy AI for Materials Discovery: From Autonomous Laboratories to Z-scores](https://mlsanity.com/paper/2512.01080/) (2025)
- [Learning Where to Simulate: Generative Active Sampling for Online PDE Surrogate Training](https://mlsanity.com/paper/2606.09949/) (2026)
- [Gaussian process surrogate with physical law-corrected prior for multi-coupled PDEs defined on irregular geometry](https://mlsanity.com/paper/2509.02617/) (2025)
- [Open-Source High-Speed Flight Surrogate Modeling Framework](https://mlsanity.com/paper/2411.03598/) (2024)
- [Estimating Parameter Fields in Multi-Physics PDEs from Scarce Measurements](https://mlsanity.com/paper/2509.00203/) (2025)
- [Performance-Carbon Trade-Offs across Architectural Biases in Shear Flow Forecasting](https://mlsanity.com/paper/2509.24517/) (2025)
- [Physics Enhanced Deep Surrogates for the Phonon Boltzmann Transport Equation](https://mlsanity.com/paper/2512.05976/) (2025)
- [Fourier Neural Operators with rank-1 lattice points and hyperbolic cross](https://mlsanity.com/paper/2606.08871/) (2026)
- [Inductively Scalable, Single-Step Neural Surrogates for Wave-Scattering Inverse Problems](https://mlsanity.com/paper/2608.17344/) (2026)
- [Cluster Attention Neural Operators for Solving Parametric Partial Differential Equations](https://mlsanity.com/paper/2609.39914/) (2026)

## Abstract (arXiv)

Many physics and engineering applications demand Partial Differential Equations (PDE) property evaluations that are traditionally computed with resource-intensive high-fidelity numerical solvers. Data-driven surrogate models provide an efficient alternative but come with a significant cost of training. Emerging applications would benefit from surrogates with an improved accuracy-cost tradeoff, while studied at scale. Here we present a "physics-enhanced deep-surrogate" ("PEDS") approach towards developing fast surrogate models for complex physical systems, which is described by PDEs. Specifically, a combination of a low-fidelity, explainable physics simulator and a neural network generator is proposed, which is trained end-to-end to globally match the output of an expensive high-fidelity numerical solver. Experiments on three exemplar testcases, diffusion, reaction-diffusion, and electromagnetic scattering models, show that a PEDS surrogate can be up to 3× more accurate than an ensemble of feedforward neural networks with limited data (≈ 103 training points), and reduces the training data need by at least a factor of 100 to achieve a target error of 5%. Experiments reveal that PEDS provides a general, data-driven strategy to bridge the gap between a vast array of simplified physical models with corresponding brute-force numerical solvers modeling complex systems, offering accuracy, speed, data efficiency, as well as physical insights into the process.