mlsanityData to 7 Oct 2026

Mamba Neural Operator: Who Wins? Transformers vs. State-Space Models for PDEs

arXiv 2410.02113 ↗First posted 3 Oct 2024cs.LG cs.NA math.NAcited by 16

On the map: State-space sequence models (S4, Mamba) → Time series forecasting with SSMs

Abstract (arXiv)

Partial differential equations (PDEs) are widely used to model complex physical systems, but solving them efficiently remains a significant challenge. Recently, Transformers have emerged as the preferred architecture for PDEs due to their ability to capture intricate dependencies. However, they struggle with representing continuous dynamics and long-range interactions. To overcome these limitations, we introduce the Mamba Neural Operator (MNO), a novel framework that enhances neural operator-based techniques for solving PDEs. MNO establishes a formal theoretical connection between structured state-space models (SSMs) and neural operators, offering a unified structure that can adapt to diverse architectures, including Transformer-based models. By leveraging the structured design of SSMs, MNO captures long-range dependencies and continuous dynamics more effectively than traditional Transformers. Through extensive analysis, we show that MNO significantly boosts the expressive power and accuracy of neural operators, making it not just a complement but a superior framework for PDE-related tasks, bridging the gap between efficient representation and accurate solution approximation. Our code is available on https://github.com/Math-ML-X/Mamba-Neural-Operator

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Time series forecasting with SSMs, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 27 of its 43 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

27 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024State-space models are accurate and efficient neural operators for dynamical systems5 Sep

Other references · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2016Layer Normalizationcited by 5,579
  5. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  6. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  7. 2024VMamba: Visual State Space Modelcited by 955
  8. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 722
  9. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  10. 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 536
  11. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 491
  12. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 490
14 more
  1. 2021Machine learning accelerated computational fluid dynamicscited by 461
  2. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 418
  3. 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 396
  4. 2021Choose a Transformer: Fourier or Galerkincited by 333
  5. 2023GNOT: A General Neural Operator Transformer for Operator Learningcited by 321
  6. 2022Transformer for Partial Differential Equations' Operator Learningcited by 270
  7. 2020Model Reduction and Neural Networks for Parametric PDEscited by 257
  8. 2021Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformerscited by 237
  9. 2021Factorized Fourier Neural Operatorscited by 237
  10. 2018Latent-space Physics: Towards Learning the Temporal Evolution of Fluid Flowcited by 97
  11. 2024On the Benefits of Memory for Modeling Time-Dependent PDEscited by 30
  12. 2024HAMLET: Graph Transformer Neural Operator for Partial Differential Equationscited by 24
  13. 2023Continuous U-Net: Faster, Greater and Noiselesscited by 8
  14. 2023The Missing U for Efficient Diffusion Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 9.

16 papers.

Sort:

By area:

  1. 2025Latent Mamba Operator for Partial Differential Equationscited by 8
  2. 2025Automatic selection of the best neural architecture for time series forecastingcited by 2
  3. 2025GeoMaNO: Geometric Mamba Neural Operator for Partial Differential Equationscited by 2
  4. 2025PDE Solvers Should Be Local: Fast, Stable Rollouts with Learned Local Stencilscited by 2
  5. 2025A Mathematical Explanation of Transformerscited by 2
  6. 2026Autoregression-Free Neural Operators for Time-Dependent PDEscited by 2
  7. 2025An Attention-based Spatio-Temporal Neural Operator for Evolving Physicscited by 1
  8. 2026Don't Fix the Basis -- Learn It: Spectral Representation with Adaptive Basis Learning for PDEscited by 1
  9. 2025Merging Memory and Space: A State Space Neural Operatorcited by 0
  10. 2026SFO: Learning PDE Operators via Spectral Filteringcited by 0
  11. 2026SpectraKAN: Conditioning Spectral Operatorscited by 0
  12. 2026CATO: Charted Attention for Neural PDE Operatorscited by 0
4 more
  1. 2026PACE-FNO: Physics-Aligned Canonical Equivariance for Fourier Neural Operatorscited by 0
  2. 2026Adaptive Mamba Neural Operatorscited by 0
  3. 2026NeuronSifter: Intervention Planning in CNS Microenvironmentscited by 0
  4. 2026Fractional Laplace Neural Operators: Exact Architectures, an Expressivity Frontier at Criticality, and Certified Stability for Memory-Driven Network Dynamicscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Stability and Discretization Error of State Space Model Neural Operators
  2. 2026Learning Physical Operators using Neural Operators
  3. 2024LE-PDE++: Mamba for accelerating PDEs Simulations
  4. 2023Hyena Neural Operator for Partial Differential Equations
  5. 2024Diffusion models as probabilistic neural operators for recovering unobserved states of dynamical systems
  6. 2025CFO: Learning Continuous-Time PDE Dynamics via Flow-Matched Neural Operators
  7. 2023Residual-Based Error Corrector Operator to Enhance Accuracy and Reliability of Neural Operator Surrogates of Nonlinear Variational Boundary-Value Problems
  8. 2026MENO: MeanFlow-Enhanced Neural Operators for Dynamical Systems