# Multiwavelet-based Operator Learning for Differential Equations

arXiv: https://arxiv.org/abs/2109.13459 · HTML: https://mlsanity.com/paper/2109.13459/

## Key facts

- Cited by 234 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 88 references that are in the map.
- First posted on arXiv 28 Sep 2021.

## Builds on (most cited first, 20 of 20)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [HiPPO: Recurrent Memory with Optimal Polynomial Projections](https://mlsanity.com/paper/2008.07669/) (2020)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/1801.06879/) (2018)
- [Symplectic Recurrent Neural Networks](https://mlsanity.com/paper/1909.13334/) (2019)
- [Solving ill-posed inverse problems using iterative deep neural networks](https://mlsanity.com/paper/1704.04058/) (2017)
- [Prediction of Aerodynamic Flow Fields Using Convolutional Neural Networks](https://mlsanity.com/paper/1905.13166/) (2019)
- [Learning to Optimize Multigrid PDE Solvers](https://mlsanity.com/paper/1902.10248/) (2019)
- [A physics-informed operator regression framework for extracting data-driven continuum models](https://mlsanity.com/paper/2009.11992/) (2020)
- [Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functions](https://mlsanity.com/paper/2105.09513/) (2021)
- [Meta-learning Pseudo-differential Operators with Deep Neural Networks](https://mlsanity.com/paper/1906.06782/) (2019)
- [Neural Vortex Method: from Finite Lagrangian Particles to Infinite Dimensional Eulerian Dynamics](https://mlsanity.com/paper/2006.04178/) (2020)
- [Design of a Simple Orthogonal Multiwavelet Filter by Matrix Spectral Factorization](https://mlsanity.com/paper/1910.07133/) (2019)
- [Algebraically-Informed Deep Networks (AIDN): A Deep Learning Approach to Represent Algebraic Structures](https://mlsanity.com/paper/2012.01141/) (2020)

## Built on by (most cited first, 30 of 234)

- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [GNOT: A General Neural Operator Transformer for Operator Learning](https://mlsanity.com/paper/2302.14376/) (2023)
- [Transolver: A Fast Transformer Solver for PDEs on General Geometries](https://mlsanity.com/paper/2402.02366/) (2024)
- [Transformer for Partial Differential Equations' Operator Learning](https://mlsanity.com/paper/2205.13671/) (2022)
- [Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers](https://mlsanity.com/paper/2111.13587/) (2021)
- [U-NO: U-shaped Neural Operators](https://mlsanity.com/paper/2204.11127/) (2022)
- [FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting](https://mlsanity.com/paper/2205.08897/) (2022)
- [Scalable Transformer for PDE Surrogate Modeling](https://mlsanity.com/paper/2305.17560/) (2023)
- [Physics-Informed Deep Neural Operator Networks](https://mlsanity.com/paper/2207.05748/) (2022)
- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [Learning Operators with Coupled Attention](https://mlsanity.com/paper/2201.01032/) (2022)
- [Solving High-Dimensional PDEs with Latent Spectral Models](https://mlsanity.com/paper/2301.12664/) (2023)
- [NOMAD: Nonlinear Manifold Decoders for Operator Learning](https://mlsanity.com/paper/2206.03551/) (2022)
- [AirfRANS: High Fidelity Computational Fluid Dynamics Dataset for Approximating Reynolds-Averaged Navier-Stokes Solutions](https://mlsanity.com/paper/2212.07564/) (2022)
- [Wavelet neural operator: a neural operator for parametric partial differential equations](https://mlsanity.com/paper/2205.02191/) (2022)
- [A Mathematical Guide to Operator Learning](https://mlsanity.com/paper/2312.14688/) (2023)
- [Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEs](https://mlsanity.com/paper/2310.00120/) (2023)
- [Learning Deep Implicit Fourier Neural Operators (IFNOs) with Applications to Heterogeneous Material Modeling](https://mlsanity.com/paper/2203.08205/) (2022)
- [Improved Operator Learning by Orthogonal Attention](https://mlsanity.com/paper/2310.12487/) (2023)
- [Learning Physical Models that Can Respect Conservation Laws](https://mlsanity.com/paper/2302.11002/) (2023)
- [CViT: Continuous Vision Transformer for Operator Learning](https://mlsanity.com/paper/2405.13998/) (2024)
- [Super-Resolution Neural Operator](https://mlsanity.com/paper/2303.02584/) (2023)
- [Mitigating spectral bias for the multiscale operator learning](https://mlsanity.com/paper/2210.10890/) (2022)
- [Operator learning with PCA-Net: upper and lower complexity bounds](https://mlsanity.com/paper/2303.16317/) (2023)
- [Nonlocality and Nonlinearity Implies Universality in Operator Learning](https://mlsanity.com/paper/2304.13221/) (2023)
- [MgNO: Efficient Parameterization of Linear Operators via Multigrid](https://mlsanity.com/paper/2310.19809/) (2023)
- [Domain Agnostic Fourier Neural Operators](https://mlsanity.com/paper/2305.00478/) (2023)
- [Parameterized Physics-informed Neural Networks for Parameterized PDEs](https://mlsanity.com/paper/2408.09446/) (2024)
- [Multilevel Diffusion: Infinite Dimensional Score-Based Diffusion Models for Image Generation](https://mlsanity.com/paper/2303.04772/) (2023)
- [Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problems](https://mlsanity.com/paper/2212.03130/) (2022)

## Abstract (arXiv)

The solution of a partial differential equation can be obtained by computing the inverse operator map between the input and the solution space. Towards this end, we introduce a textitmultiwavelet-based neural operator learning scheme that compresses the associated operator's kernel using fine-grained wavelets. By explicitly embedding the inverse multiwavelet filters, we learn the projection of the kernel onto fixed multiwavelet polynomial bases. The projected kernel is trained at multiple scales derived from using repeated computation of multiwavelet transform. This allows learning the complex dependencies at various scales and results in a resolution-independent scheme. Compare to the prior works, we exploit the fundamental properties of the operator's kernel which enable numerically efficient representation. We perform experiments on the Korteweg-de Vries (KdV) equation, Burgers' equation, Darcy Flow, and Navier-Stokes equation. Compared with the existing neural operator approaches, our model shows significantly higher accuracy and achieves state-of-the-art in a range of datasets. For the time-varying equations, the proposed method exhibits a (2X-10X) improvement (0.0018 (0.0033) relative L2 error for Burgers' (KdV) equation). By learning the mappings between function spaces, the proposed method has the ability to find the solution of a high-resolution input after learning from lower-resolution data.