# SVD Perspectives for Augmenting DeepONet Flexibility and Interpretability

arXiv: https://arxiv.org/abs/2204.12670 · HTML: https://mlsanity.com/paper/2204.12670/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 100 references that are in the map.
- First posted on arXiv 27 Apr 2022.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [Machine Learning for Fluid Mechanics](https://mlsanity.com/paper/1905.11075/) (2019)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [MIONet: Learning multiple-input operators via tensor product](https://mlsanity.com/paper/2202.06137/) (2022)
- [A physics-informed variational DeepONet for predicting the crack path in brittle materials](https://mlsanity.com/paper/2108.06905/) (2021)
- [Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems](https://mlsanity.com/paper/1912.08177/) (2019)
- [Multifidelity deep neural operators for efficient learning of partial differential equations with application to fast inverse design of nanoscale heat transport](https://mlsanity.com/paper/2204.06684/) (2022)
- [Learning Operators with Coupled Attention](https://mlsanity.com/paper/2201.01032/) (2022)
- [Long-time integration of parametric evolution equations with physics-informed DeepONets](https://mlsanity.com/paper/2106.05384/) (2021)
- [Improved architectures and training algorithms for deep operator networks](https://mlsanity.com/paper/2110.01654/) (2021)
- [A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder](https://mlsanity.com/paper/2009.11990/) (2020)
- [Learning two-phase microstructure evolution using neural operators and autoencoder architectures](https://mlsanity.com/paper/2204.07230/) (2022)
- [Learning Deep Implicit Fourier Neural Operators (IFNOs) with Applications to Heterogeneous Material Modeling](https://mlsanity.com/paper/2203.08205/) (2022)
- [DeepGreen: Deep Learning of Green's Functions for Nonlinear Boundary Value Problems](https://mlsanity.com/paper/2101.07206/) (2020)
- [A physics-informed operator regression framework for extracting data-driven continuum models](https://mlsanity.com/paper/2009.11992/) (2020)
- [Nonlocal Kernel Network (NKN): a Stable and Resolution-Independent Deep Neural Network](https://mlsanity.com/paper/2201.02217/) (2022)
- [Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations](https://mlsanity.com/paper/2112.08125/) (2021)
- [Simulating progressive intramural damage leading to aortic dissection using an operator-regression neural network](https://mlsanity.com/paper/2108.11985/) (2021)
- [On the influence of over-parameterization in manifold based surrogates and deep neural operators](https://mlsanity.com/paper/2203.05071/) (2022)
- [Bi-fidelity Modeling of Uncertain and Partially Unknown Systems using DeepONets](https://mlsanity.com/paper/2204.00997/) (2022)
- [GMLS-Nets: A framework for learning from unstructured data](https://mlsanity.com/paper/1909.05371/) (2019)
- [The Neural Network shifted-Proper Orthogonal Decomposition: a Machine Learning Approach for Non-linear Reduction of Hyperbolic Equations](https://mlsanity.com/paper/2108.06558/) (2021)
- [MultiAuto-DeepONet: A Multi-resolution Autoencoder DeepONet for Nonlinear Dimension Reduction, Uncertainty Quantification and Operator Learning of Forward and Inverse Stochastic Problems](https://mlsanity.com/paper/2204.03193/) (2022)
- [Enhanced DeepONet for Modeling Partial Differential Operators Considering Multiple Input Functions](https://mlsanity.com/paper/2202.08942/) (2022)

## Built on by (most cited first, 30 of 35)

- [A Mathematical Guide to Operator Learning](https://mlsanity.com/paper/2312.14688/) (2023)
- [CFDBench: A Large-Scale Benchmark for Machine Learning Methods in Fluid Dynamics](https://mlsanity.com/paper/2310.05963/) (2023)
- [Transformers as Neural Operators for Solutions of Differential Equations with Finite Regularity](https://mlsanity.com/paper/2405.19166/) (2024)
- [On the training and generalization of deep operator networks](https://mlsanity.com/paper/2309.01020/) (2023)
- [Learning stiff chemical kinetics using extended deep neural operators](https://mlsanity.com/paper/2302.12645/) (2023)
- [RiemannONets: Interpretable Neural Operators for Riemann Problems](https://mlsanity.com/paper/2401.08886/) (2024)
- [Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networks](https://mlsanity.com/paper/2205.02902/) (2022)
- [HyperDeepONet: learning operator with complex target function space using the limited resources via hypernetwork](https://mlsanity.com/paper/2312.15949/) (2023)
- [RandONet: Shallow-Networks with Random Projections for learning linear and nonlinear operators](https://mlsanity.com/paper/2406.05470/) (2024)
- [Deciphering and integrating invariants for neural operator learning with various physical mechanisms](https://mlsanity.com/paper/2311.14361/) (2023)
- [Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation](https://mlsanity.com/paper/2302.05104/) (2023)
- [Adaptive physics-informed neural operator for coarse-grained non-equilibrium flows](https://mlsanity.com/paper/2210.15799/) (2022)
- [PINN surrogate of Li-ion battery models for parameter inference. Part I: Implementation and multi-fidelity hierarchies for the single-particle model](https://mlsanity.com/paper/2312.17329/) (2023)
- [Separable Operator Networks](https://mlsanity.com/paper/2407.11253/) (2024)
- [PeSANet: Physics-encoded Spectral Attention Network for Simulating PDE-Governed Complex Systems](https://mlsanity.com/paper/2505.01736/) (2025)
- [FD-Bench: A Modular and Fair Benchmark for Data-driven Fluid Simulation](https://mlsanity.com/paper/2505.20349/) (2025)
- [Solving Partial Differential Equations in Different Domains by Operator Learning method Based on Boundary Integral Equations](https://mlsanity.com/paper/2406.02298/) (2024)
- [Estimating Ground Reaction Forces from Inertial Sensors](https://mlsanity.com/paper/2311.02287/) (2023)
- [MD-PNOP: Equation-Recast Neural Operators for Minimal-Data Extrapolation and PDE Solver Acceleration](https://mlsanity.com/paper/2509.01416/) (2025)
- [AMORE: Adaptive Multi-Output Operator Network for Stiff Chemical Kinetics](https://mlsanity.com/paper/2510.12999/) (2025)
- [Learning Data-Efficient and Generalizable Neural Operators via Fundamental Physics Knowledge](https://mlsanity.com/paper/2602.15184/) (2026)
- [Using Neural Implicit Flow To Represent Latent Dynamics Of Canonical Systems](https://mlsanity.com/paper/2404.17535/) (2024)
- [PinnDE: Physics-Informed Neural Networks for Solving Differential Equations](https://mlsanity.com/paper/2408.10011/) (2024)
- [Invertible Koopman neural operator for data-driven modeling of partial differential equations](https://mlsanity.com/paper/2503.19717/) (2025)
- [Equation Recast for Canonical Operator Learning Across Parametric PDEs](https://mlsanity.com/paper/2609.02982/) (2026)
- [A Framework for Combustion Chemistry Acceleration with DeepONets](https://mlsanity.com/paper/2304.12188/) (2023)
- [Is Deep Learning Network Necessary for Image Generation?](https://mlsanity.com/paper/2308.13612/) (2023)
- [Low-rank adaptive physics-informed HyperDeepONets for solving differential equations](https://mlsanity.com/paper/2507.18346/) (2025)
- [Hybrid twinning using PBDW and DeepONet for the effective state estimation and prediction on partially known systems](https://mlsanity.com/paper/2512.11834/) (2025)
- [Kinetic-Mamba: Mamba-Assisted Predictions of Stiff Chemical Kinetics](https://mlsanity.com/paper/2512.14471/) (2025)

## Abstract (arXiv)

Deep operator networks (DeepONets) are powerful architectures for fast and accurate emulation of complex dynamics. As their remarkable generalization capabilities are primarily enabled by their projection-based attribute, we investigate connections with low-rank techniques derived from the singular value decomposition (SVD). We demonstrate that some of the concepts behind proper orthogonal decomposition (POD)-neural networks can improve DeepONet's design and training phases. These ideas lead us to a methodology extension that we name SVD-DeepONet. Moreover, through multiple SVD analyses, we find that DeepONet inherits from its projection-based attribute strong inefficiencies in representing dynamics characterized by symmetries. Inspired by the work on shifted-POD, we develop flexDeepONet, an architecture enhancement that relies on a pre-transformation network for generating a moving reference frame and isolating the rigid components of the dynamics. In this way, the physics can be represented on a latent space free from rotations, translations, and stretches, and an accurate projection can be performed to a low-dimensional basis. In addition to flexibility and interpretability, the proposed perspectives increase DeepONet's generalization capabilities and computational efficiencies. For instance, we show flexDeepONet can accurately surrogate the dynamics of 19 variables in a combustion chemistry application by relying on 95% less trainable parameters than the ones of the vanilla architecture. We argue that DeepONet and SVD-based methods can reciprocally benefit from each other. In particular, the flexibility of the former in leveraging multiple data sources and multifidelity knowledge in the form of both unstructured data and physics-informed constraints has the potential to greatly extend the applicability of methodologies such as POD and PCA.