# Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEs

arXiv: https://arxiv.org/abs/2310.00120 · HTML: https://mlsanity.com/paper/2310.00120/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 65 references that are in the map.
- First posted on arXiv 29 Sep 2023.

## Builds on (most cited first, 30 of 36)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flow](https://mlsanity.com/paper/2109.03697/) (2021)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Tensorizing Neural Networks](https://mlsanity.com/paper/1509.06569/) (2015)
- [Factorized Fourier Neural Operators](https://mlsanity.com/paper/2111.13802/) (2021)
- [Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers](https://mlsanity.com/paper/2111.13587/) (2021)
- [Multiwavelet-based Operator Learning for Differential Equations](https://mlsanity.com/paper/2109.13459/) (2021)
- [U-NO: U-shaped Neural Operators](https://mlsanity.com/paper/2204.11127/) (2022)
- [Tensor Decomposition for Signal Processing and Machine Learning](https://mlsanity.com/paper/1607.01668/) (2016)
- [Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition](https://mlsanity.com/paper/1412.6553/) (2014)
- [Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/1801.06879/) (2018)
- [Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications](https://mlsanity.com/paper/1511.06530/) (2015)
- [Solving ill-posed inverse problems using iterative deep neural networks](https://mlsanity.com/paper/1704.04058/) (2017)
- [Prediction of Aerodynamic Flow Fields Using Convolutional Neural Networks](https://mlsanity.com/paper/1905.13166/) (2019)
- [Sobolev Training for Neural Networks](https://mlsanity.com/paper/1706.04859/) (2017)
- [Monarch: Expressive Structured Matrices for Efficient and Accurate Training](https://mlsanity.com/paper/2204.00595/) (2022)
- [TensorLy: Tensor Learning in Python](https://mlsanity.com/paper/1610.09555/) (2016)
- [MeshfreeFlowNet: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework](https://mlsanity.com/paper/2005.01463/) (2020)
- [Multi-Head Attention: Collaborate Instead of Concatenate](https://mlsanity.com/paper/2006.16362/) (2020)
- [Tensor Methods in Computer Vision and Deep Learning](https://mlsanity.com/paper/2107.03436/) (2021)

## Built on by (most cited first, 30 of 68)

- [The Well: a Large-Scale Collection of Diverse Physics Simulations for Machine Learning](https://mlsanity.com/paper/2412.00568/) (2024)
- [Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs](https://mlsanity.com/paper/2403.12553/) (2024)
- [A Library for Learning Neural Operators](https://mlsanity.com/paper/2412.10354/) (2024)
- [Plasma Surrogate Modelling using Fourier Neural Operators](https://mlsanity.com/paper/2311.05967/) (2023)
- [Generative Latent Neural PDE Solver using Flow Matching](https://mlsanity.com/paper/2503.22600/) (2025)
- [Linearization Turns Neural Operators into Function-Valued Gaussian Processes](https://mlsanity.com/paper/2406.05072/) (2024)
- [Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learning](https://mlsanity.com/paper/2506.10973/) (2025)
- [Kernel Neural Operators (KNOs) for Scalable, Memory-efficient, Geometrically-flexible Operator Learning](https://mlsanity.com/paper/2407.00809/) (2024)
- [Predicting Change, Not States: An Alternate Framework for Neural PDE Surrogates](https://mlsanity.com/paper/2412.13074/) (2024)
- [TANTE: Time-Adaptive Operator Learning via Neural Taylor Expansion](https://mlsanity.com/paper/2502.08574/) (2025)
- [AMR-Transformer: Enabling Efficient Long-range Interaction for Complex Neural Fluid Simulation](https://mlsanity.com/paper/2503.10257/) (2025)
- [FEDONet : Fourier-Embedded DeepONet for Spectrally Accurate Operator Learning](https://mlsanity.com/paper/2509.12344/) (2025)
- [FreqMoE: Dynamic Frequency Enhancement for Neural PDE Solvers](https://mlsanity.com/paper/2505.06858/) (2025)
- [Performance of Neural and Polynomial Operator Surrogates](https://mlsanity.com/paper/2604.00689/) (2026)
- [TensorGRaD: Tensor Gradient Robust Decomposition for Memory-Efficient Neural Operator Training](https://mlsanity.com/paper/2501.02379/) (2025)
- [Light-Weight Diffusion Multiplier and Uncertainty Quantification for Fourier Neural Operators](https://mlsanity.com/paper/2508.00643/) (2025)
- [EddyFormer: Accelerated Neural Simulations of Three-Dimensional Turbulence at Scale](https://mlsanity.com/paper/2510.24173/) (2025)
- [Spectral Embedding via Chebyshev Bases for Robust DeepONet Approximation](https://mlsanity.com/paper/2512.09165/) (2025)
- [Learning PDE Solvers with Physics and Data: A Unifying View of Physics-Informed Neural Networks and Neural Operators](https://mlsanity.com/paper/2601.14517/) (2026)
- [Separable Operator Networks](https://mlsanity.com/paper/2407.11253/) (2024)
- [PACE: Pacing Operator Learning to Accurate Optical Field Simulation for Complicated Photonic Devices](https://mlsanity.com/paper/2411.03527/) (2024)
- [Overtone: Cyclic Patch Modulation for Clean, Efficient, and Flexible Physics Emulators](https://mlsanity.com/paper/2507.09264/) (2025)
- [PEST: Physics-Enhanced Swin Transformer for 3D Turbulence Simulation](https://mlsanity.com/paper/2602.10150/) (2026)
- [Global-local Fourier Neural Operator for Accelerating Coronal Magnetic Field Model](https://mlsanity.com/paper/2405.12754/) (2024)
- [Simple yet Effective: Low-Rank Spatial Attention for Neural Operators](https://mlsanity.com/paper/2604.03582/) (2026)
- [Emergent Transfer of a Physics Foundation Model from Simulation to Laboratory Turbulence](https://mlsanity.com/paper/2606.01470/) (2026)
- [LaDEEP: A Deep Learning-based Surrogate Model for Large Deformation of Elastic-Plastic Solids](https://mlsanity.com/paper/2506.06001/) (2025)
- [Method of Manufactured Learning for Solver-free Training of Neural Operators](https://mlsanity.com/paper/2511.12890/) (2025)
- [WorldParticle: Unified World Simulation of Lagrangian Particle Dynamics via Transformer](https://mlsanity.com/paper/2605.15305/) (2026)
- [Fourier Neural Operators for Learning Dynamics in Quantum Spin Systems](https://mlsanity.com/paper/2409.03302/) (2024)

## Abstract (arXiv)

Memory complexity and data scarcity have so far prohibited learning solution operators of partial differential equations (PDEs) at high resolutions. We address these limitations by introducing a new data efficient and highly parallelizable operator learning approach with reduced memory requirement and better generalization, called multi-grid tensorized neural operator (MG-TFNO). MG-TFNO scales to large resolutions by leveraging local and global structures of full-scale, real-world phenomena, through a decomposition of both the input domain and the operator's parameter space. Our contributions are threefold: i) we enable parallelization over input samples with a novel multi-grid-based domain decomposition, ii) we represent the parameters of the model in a high-order latent subspace of the Fourier domain, through a global tensor factorization, resulting in an extreme reduction in the number of parameters and improved generalization, and iii) we propose architectural improvements to the backbone FNO. Our approach can be used in any operator learning setting. We demonstrate superior performance on the turbulent Navier-Stokes equations where we achieve less than half the error with over 150x compression. The tensorization combined with the domain decomposition, yields over 150x reduction in the number of parameters and 7x reduction in the domain size without losses in accuracy, while slightly enabling parallelism.