# How important are specialized transforms in Neural Operators?

arXiv: https://arxiv.org/abs/2308.09293 · HTML: https://mlsanity.com/paper/2308.09293/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 15 references that are in the map.
- First posted on arXiv 18 Aug 2023.

## Builds on (most cited first, 8 of 8)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Multiwavelet-based Operator Learning for Differential Equations](https://mlsanity.com/paper/2109.13459/) (2021)
- [LNO: Laplace Neural Operator for Solving Differential Equations](https://mlsanity.com/paper/2303.10528/) (2023)
- [Physics informed WNO](https://mlsanity.com/paper/2302.05925/) (2023)
- [Universal Approximation Theorem for Neural Networks](https://mlsanity.com/paper/2102.10993/) (2021)
- [Deep BSDE-ML Learning and Its Application to Model-Free Optimal Control](https://mlsanity.com/paper/2201.01318/) (2022)

## Built on by (most cited first, 2 of 2)

- [Can Physics Informed Neural Operators Self Improve?](https://mlsanity.com/paper/2311.13885/) (2023)
- [A novel Fourier neural operator framework for classification of multi-sized images: Application to three dimensional digital porous media](https://mlsanity.com/paper/2402.11568/) (2024)

## Abstract (arXiv)

Simulating physical systems using Partial Differential Equations (PDEs) has become an indispensible part of modern industrial process optimization. Traditionally, numerical solvers have been used to solve the associated PDEs, however recently Transform-based Neural Operators such as the Fourier Neural Operator and Wavelet Neural Operator have received a lot of attention for their potential to provide fast solutions for systems of PDEs. In this work, we investigate the importance of the transform layers to the reported success of transform based neural operators. In particular, we record the cost in terms of performance, if all the transform layers are replaced by learnable linear layers. Surprisingly, we observe that linear layers suffice to provide performance comparable to the best-known transform-based layers and seem to do so with a compute time advantage as well. We believe that this observation can have significant implications for future work on Neural Operators, and might point to other sources of efficiencies for these architectures.