# Mamba Neural Operator: Who Wins? Transformers vs. State-Space Models for PDEs

arXiv: https://arxiv.org/abs/2410.02113 · HTML: https://mlsanity.com/paper/2410.02113/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Time series forecasting with SSMs, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 27 of its 43 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Time series forecasting with SSMs](https://mlsanity.com/ssm/time-series-forecasting-with-ssms/)

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Gated Linear Attention Transformers with Hardware-Efficient Training](https://mlsanity.com/paper/2312.06635/) (2023)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [On the Parameterization and Initialization of Diagonal State Space Models](https://mlsanity.com/paper/2206.11893/) (2022)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [Choose a Transformer: Fourier or Galerkin](https://mlsanity.com/paper/2105.14995/) (2021)
- [GNOT: A General Neural Operator Transformer for Operator Learning](https://mlsanity.com/paper/2302.14376/) (2023)
- [Transformer for Partial Differential Equations' Operator Learning](https://mlsanity.com/paper/2205.13671/) (2022)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Factorized Fourier Neural Operators](https://mlsanity.com/paper/2111.13802/) (2021)
- [Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers](https://mlsanity.com/paper/2111.13587/) (2021)
- [Latent-space Physics: Towards Learning the Temporal Evolution of Fluid Flow](https://mlsanity.com/paper/1802.10123/) (2018)
- [On the Benefits of Memory for Modeling Time-Dependent PDEs](https://mlsanity.com/paper/2409.02313/) (2024)
- [State-space models are accurate and efficient neural operators for dynamical systems](https://mlsanity.com/paper/2409.03231/) (2024)
- [HAMLET: Graph Transformer Neural Operator for Partial Differential Equations](https://mlsanity.com/paper/2402.03541/) (2024)
- [Continuous U-Net: Faster, Greater and Noiseless](https://mlsanity.com/paper/2302.00626/) (2023)
- [The Missing U for Efficient Diffusion Models](https://mlsanity.com/paper/2310.20092/) (2023)

## Built on by (most cited first, 16 of 16)

- [Latent Mamba Operator for Partial Differential Equations](https://mlsanity.com/paper/2505.19105/) (2025)
- [Automatic selection of the best neural architecture for time series forecasting](https://mlsanity.com/paper/2501.12215/) (2025)
- [GeoMaNO: Geometric Mamba Neural Operator for Partial Differential Equations](https://mlsanity.com/paper/2505.12020/) (2025)
- [Merging Memory and Space: A State Space Neural Operator](https://mlsanity.com/paper/2507.23428/) (2025)
- [PDE Solvers Should Be Local: Fast, Stable Rollouts with Learned Local Stencils](https://mlsanity.com/paper/2509.26186/) (2025)
- [A Mathematical Explanation of Transformers](https://mlsanity.com/paper/2510.03989/) (2025)
- [Autoregression-Free Neural Operators for Time-Dependent PDEs](https://mlsanity.com/paper/2605.25413/) (2026)
- [An Attention-based Spatio-Temporal Neural Operator for Evolving Physics](https://mlsanity.com/paper/2506.11328/) (2025)
- [Don't Fix the Basis -- Learn It: Spectral Representation with Adaptive Basis Learning for PDEs](https://mlsanity.com/paper/2605.10451/) (2026)
- [Adaptive Mamba Neural Operators](https://mlsanity.com/paper/2607.18043/) (2026)
- [SFO: Learning PDE Operators via Spectral Filtering](https://mlsanity.com/paper/2601.17090/) (2026)
- [SpectraKAN: Conditioning Spectral Operators](https://mlsanity.com/paper/2602.05187/) (2026)
- [CATO: Charted Attention for Neural PDE Operators](https://mlsanity.com/paper/2605.09016/) (2026)
- [PACE-FNO: Physics-Aligned Canonical Equivariance for Fourier Neural Operators](https://mlsanity.com/paper/2605.18606/) (2026)
- [NeuronSifter: An Agent-in-Twin for Prioritizing Central Nervous System Interventions with World Action Models](https://mlsanity.com/paper/2609.35445/) (2026)
- [Fractional Laplace Neural Operators: Exact Architectures, an Expressivity Frontier at Criticality, and Certified Stability for Memory-Driven Network Dynamics](https://mlsanity.com/paper/2610.00515/) (2026)

## Abstract (arXiv)

Partial differential equations (PDEs) are widely used to model complex physical systems, but solving them efficiently remains a significant challenge. Recently, Transformers have emerged as the preferred architecture for PDEs due to their ability to capture intricate dependencies. However, they struggle with representing continuous dynamics and long-range interactions. To overcome these limitations, we introduce the Mamba Neural Operator (MNO), a novel framework that enhances neural operator-based techniques for solving PDEs. MNO establishes a formal theoretical connection between structured state-space models (SSMs) and neural operators, offering a unified structure that can adapt to diverse architectures, including Transformer-based models. By leveraging the structured design of SSMs, MNO captures long-range dependencies and continuous dynamics more effectively than traditional Transformers. Through extensive analysis, we show that MNO significantly boosts the expressive power and accuracy of neural operators, making it not just a complement but a superior framework for PDE-related tasks, bridging the gap between efficient representation and accurate solution approximation. Our code is available on https://github.com/Math-ML-X/Mamba-Neural-Operator