# A Mathematical Guide to Operator Learning

arXiv: https://arxiv.org/abs/2312.14688 · HTML: https://mlsanity.com/paper/2312.14688/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- Builds on 70 of its 154 references that are in the map.
- First posted on arXiv 22 Dec 2023.

## Builds on (most cited first, 30 of 70)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [A Convergence Theory for Deep Learning via Over-Parameterization](https://mlsanity.com/paper/1811.03962/) (2018)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Gradient Descent Finds Global Minima of Deep Neural Networks](https://mlsanity.com/paper/1811.03804/) (2018)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Fourier Neural Operator with Learned Deformations for PDEs on General Geometries](https://mlsanity.com/paper/2207.05209/) (2022)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [Choose a Transformer: Fourier or Galerkin](https://mlsanity.com/paper/2105.14995/) (2021)

## Built on by (most cited first, 30 of 76)

- [DISCO: learning to DISCover an evolution Operator for multi-physics-agnostic prediction](https://mlsanity.com/paper/2504.19496/) (2025)
- [Operator Learning with Gaussian Processes](https://mlsanity.com/paper/2409.04538/) (2024)
- [Using Uncertainty Quantification to Characterize and Improve Out-of-Domain Learning for PDEs](https://mlsanity.com/paper/2403.10642/) (2024)
- [A finite element-based physics-informed operator learning framework for spatiotemporal partial differential equations on arbitrary domains](https://mlsanity.com/paper/2405.12465/) (2024)
- [CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations](https://mlsanity.com/paper/2402.14646/) (2024)
- [Linearization Turns Neural Operators into Function-Valued Gaussian Processes](https://mlsanity.com/paper/2406.05072/) (2024)
- [Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learning](https://mlsanity.com/paper/2506.10973/) (2025)
- [Spectral-Refiner: Accurate Fine-Tuning of Spatiotemporal Fourier Neural Operator for Turbulent Flows](https://mlsanity.com/paper/2405.17211/) (2024)
- [The Sample Complexity of Learning Lipschitz Operators with respect to Gaussian Measures](https://mlsanity.com/paper/2410.23440/) (2024)
- [Learning the Hodgkin-Huxley Model with Operator Learning Techniques](https://mlsanity.com/paper/2406.02173/) (2024)
- [Enhancing Fourier Neural Operators with Local Spatial Features](https://mlsanity.com/paper/2503.17797/) (2025)
- [Test-time Generalization for Physics through Neural Operator Splitting](https://mlsanity.com/paper/2602.00884/) (2026)
- [Operator learning without the adjoint](https://mlsanity.com/paper/2401.17739/) (2024)
- [Optimal Convergence Rates for Neural Operators](https://mlsanity.com/paper/2412.17518/) (2024)
- [Data-Efficient Kernel Methods for Learning Differential Equations and Their Solution Operators: Algorithms and Error Analysis](https://mlsanity.com/paper/2503.01036/) (2025)
- [Operator Learning at Machine Precision](https://mlsanity.com/paper/2511.19980/) (2025)
- [An Advanced Physics-Informed Neural Operator for Comprehensive Design Optimization of Highly-Nonlinear Systems: An Aerospace Composites Processing Case Study](https://mlsanity.com/paper/2406.14715/) (2024)
- [Optimal sampling for least-squares approximation](https://mlsanity.com/paper/2409.02342/) (2024)
- [Scalable Mechanistic Neural Networks for Differential Equations and Machine Learning](https://mlsanity.com/paper/2410.06074/) (2024)
- [Towards Sharp Minimax Risk Bounds for Operator Learning](https://mlsanity.com/paper/2512.17805/) (2025)
- [Multi-Level Monte Carlo Training of Neural Operators](https://mlsanity.com/paper/2505.12940/) (2025)
- [A short tour of operator learning theory: Convergence rates, statistical limits, and open questions](https://mlsanity.com/paper/2603.00819/) (2026)
- [One Operator for Many Densities: Amortized Approximation of Conditioning by Neural Operators](https://mlsanity.com/paper/2605.06873/) (2026)
- [Topological Neural Operators](https://mlsanity.com/paper/2606.09806/) (2026)
- [Error analysis for finite element operator learning methods for solving parametric second-order elliptic PDEs](https://mlsanity.com/paper/2404.17868/) (2024)
- [Green Multigrid Network](https://mlsanity.com/paper/2407.03593/) (2024)
- [Structure-Preserving Operator Learning](https://mlsanity.com/paper/2410.01065/) (2024)
- [Learning sparsity-promoting regularizers for linear inverse problems](https://mlsanity.com/paper/2412.16031/) (2024)
- [Orthogonal greedy algorithm for linear operator learning with shallow neural network](https://mlsanity.com/paper/2501.02791/) (2025)
- [Uncertainty propagation in feed-forward neural network models](https://mlsanity.com/paper/2503.21059/) (2025)

## Abstract (arXiv)

Operator learning aims to discover properties of an underlying dynamical system or partial differential equation (PDE) from data. Here, we present a step-by-step guide to operator learning. We explain the types of problems and PDEs amenable to operator learning, discuss various neural network architectures, and explain how to employ numerical PDE solvers effectively. We also give advice on how to create and manage training data and conduct optimization. We offer intuition behind the various neural network architectures employed in operator learning by motivating them from the point-of-view of numerical linear algebra.