mlsanityData to 7 Oct 2026

State-space models are accurate and efficient neural operators for dynamical systems

arXiv 2409.03231 ↗First posted 5 Sep 2024cs.LG cs.NA math.DScited by 27

On the map: State-space sequence models (S4, Mamba) → Time series forecasting with SSMs

Abstract (arXiv)

Physics-informed machine learning (PIML) has emerged as a promising alternative to classical methods for predicting dynamical systems, offering faster and more generalizable solutions. However, existing models, including recurrent neural networks (RNNs), transformers, and neural operators, face challenges such as long-time integration, long-range dependencies, chaotic dynamics, and extrapolation, to name a few. To this end, this paper introduces state-space models implemented in Mamba for accurate and efficient dynamical system operator learning. Mamba addresses the limitations of existing architectures by dynamically capturing long-range dependencies and enhancing computational efficiency through reparameterization techniques. To extensively test Mamba and compare against another 11 baselines, we introduce several strict extrapolation testbeds that go beyond the standard interpolation benchmarks. We demonstrate Mamba's superior performance in both interpolation and challenging extrapolation tasks. Mamba consistently ranks among the top models while maintaining the lowest computational cost and exceptional extrapolation capabilities. Moreover, we demonstrate the good performance of Mamba for a real-world application in quantitative systems pharmacology for assessing the efficacy of drugs in tumor growth under limited data scenarios. Taken together, our findings highlight Mamba's potential as a powerful tool for advancing scientific machine learning in dynamical systems modeling. (The code will be available at https://github.com/zheyuanhu01/State_Space_Model_Neural_Operator upon acceptance.)

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in Time series forecasting with SSMs, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 40 of its 58 references that are in the map. First posted on arXiv 5 Sep 2024.

Builds on

40 of 58 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2016Layer Normalizationcited by 5,579
  8. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  9. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  10. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  11. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  12. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
28 more
  1. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  2. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  3. 2018Group Normalizationcited by 1,514
  4. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  5. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  6. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 594
  7. 2021Learning the solution operator of parametric partial differential equations with physics-informed DeepOnetscited by 537
  8. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 495
  9. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 388
  10. 2021Choose a Transformer: Fourier or Galerkincited by 333
  11. 2023GNOT: A General Neural Operator Transformer for Operator Learningcited by 321
  12. 2022Transformer for Partial Differential Equations' Operator Learningcited by 270
  13. 2022Long Range Language Modeling via Gated State Spacescited by 265
  14. 2023HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolutioncited by 187
  15. 2019Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamicscited by 132
  16. 2022Physics-Informed Deep Neural Operator Networkscited by 110
  17. 2021A physics-informed variational DeepONet for predicting the crack path in brittle materialscited by 98
  18. 2023Tackling the Curse of Dimensionality with Physics-Informed Neural Networkscited by 78
  19. 2022Reliable extrapolation of deep neural operators informed by physics or sparse observationscited by 62
  20. 2018Can recurrent neural networks warp time?cited by 53
  21. 2022Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodologycited by 43
  22. 2021Learning Dissipative Dynamics in Chaotic Systemscited by 41
  23. 2024Transformers as Neural Operators for Solutions of Differential Equations with Finite Regularitycited by 33
  24. 2023Neural Operator Learning for Long-Time Integration in Dynamical Systems with Recurrent Neural Networkscited by 32
  25. 2023Learning stiff chemical kinetics using extended deep neural operatorscited by 30
  26. 2022Physics guided neural networks for modelling of non-linear dynamicscited by 20
  27. 2023Deep neural operators can serve as accurate surrogates for shape optimization: A case study for airfoilscited by 19
  28. 2020Physics-Incorporated Convolutional Recurrent Neural Networks for Source Identification and Forecasting of Dynamical Systemscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 19; 2026: 4.

27 papers.

Sort:

By area:

  1. 2024Oscillatory State-Space Modelscited by 35
  2. 2024Mamba Neural Operator: Who Wins? Transformers vs. State-Space Models for PDEscited by 16
  3. 2025Recurrent Neural Operators: Stable Long-Term PDE Predictioncited by 15
  4. 2025Comba: Improving Bilinear RNNs with Closed-loop Controlcited by 12
  5. 2025Sub-Sequential Physics-Informed Learning with State Space Modelcited by 10
  6. 2025Representation Meets Optimization: Training PINNs and PIKANs for Gray-Box Discovery in Systems Pharmacologycited by 10
  7. 2025Temporal Neural Operator for Modeling Time-Dependent Physical Phenomenacited by 8
  8. 2025Latent Mamba Operator for Partial Differential Equationscited by 8
  9. 2024CMINNs: Compartment Model Informed Neural Networks -- Unlocking Drug Dynamicscited by 7
  10. 2025Block-Biased Mamba for Long-Range Sequence Processingcited by 7
  11. 2025Learning to Dissipate Energy in Oscillatory State-Space Modelscited by 7
  12. 2026Learning PDE Solvers with Physics and Data: A Unifying View of Physics-Informed Neural Networks and Neural Operatorscited by 6
15 more
  1. 2025Physics-Informed Machine Learning in Biomedical Science and Engineeringcited by 4
  2. 2025Automatic selection of the best neural architecture for time series forecastingcited by 2
  3. 2025GeoMaNO: Geometric Mamba Neural Operator for Partial Differential Equationscited by 2
  4. 2025AMORE: Adaptive Multi-Output Operator Network for Stiff Chemical Kineticscited by 2
  5. 2026Geometric Neural Operators via Lie Group-Constrained Latent Dynamicscited by 2
  6. 2025FMaMIL: Frequency-Driven Mamba Multi-Instance Learning for Weakly Supervised Lesion Segmentation in Medical Imagescited by 1
  7. 2025Neuronal Group Communication for Efficient Neural representationcited by 1
  8. 2024Variational autoencoders with latent high-dimensional steady geometric flows for dynamicscited by 0
  9. 2025Drivetrain simulation using variational autoencoderscited by 0
  10. 2025Observability conditions for neural state-space models with eigenvalues and their roots of unitycited by 0
  11. 2025Merging Memory and Space: A State Space Neural Operatorcited by 0
  12. 2025ASecond-Order SpikingSSM for Wearablescited by 0
  13. 2025Kinetic-Mamba: Mamba-Assisted Predictions of Stiff Chemical Kineticscited by 0
  14. 2026Time-Delayed Transformers for Data-Driven Modeling of Low-Dimensional Dynamicscited by 0
  15. 2026Adaptive Mamba Neural Operatorscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Physics-guided Deep Markov Models for Learning Nonlinear Dynamical Systems with Uncertainty
  2. 2024A Survey of Mamba
  3. 2025Trained Mamba Emulates Online Gradient Descent in In-Context Linear Regression
  4. 2026Graph Mamba Operator: A Latent Simulator for Interacting Particle Systems
  5. 2026A Theoretical Analysis of Mamba's Training Dynamics: Filtering Relevant Features for Generalization in State Space Models
  6. 2024Demystifying the Token Dynamics of Deep Selective State Space Models
  7. 2024HiPPO-Prophecy: State-Space Models can Provably Learn Dynamical Systems in Context
  8. 2024Mamba State-Space Models Are Lyapunov-Stable Learners