# State-space models are accurate and efficient neural operators for dynamical systems

arXiv: https://arxiv.org/abs/2409.03231 · HTML: https://mlsanity.com/paper/2409.03231/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Time series forecasting with SSMs, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 40 of its 58 references that are in the map.
- First posted on arXiv 5 Sep 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Time series forecasting with SSMs](https://mlsanity.com/ssm/time-series-forecasting-with-ssms/)

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [RWKV: Reinventing RNNs for the Transformer Era](https://mlsanity.com/paper/2305.13048/) (2023)
- [Retentive Network: A Successor to Transformer for Large Language Models](https://mlsanity.com/paper/2307.08621/) (2023)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Hungry Hungry Hippos: Towards Language Modeling with State Space Models](https://mlsanity.com/paper/2212.14052/) (2022)
- [Hyena Hierarchy: Towards Larger Convolutional Language Models](https://mlsanity.com/paper/2302.10866/) (2023)
- [Choose a Transformer: Fourier or Galerkin](https://mlsanity.com/paper/2105.14995/) (2021)
- [GNOT: A General Neural Operator Transformer for Operator Learning](https://mlsanity.com/paper/2302.14376/) (2023)
- [Transformer for Partial Differential Equations' Operator Learning](https://mlsanity.com/paper/2205.13671/) (2022)
- [Long Range Language Modeling via Gated State Spaces](https://mlsanity.com/paper/2206.13947/) (2022)
- [HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution](https://mlsanity.com/paper/2306.15794/) (2023)
- [Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/1910.05266/) (2019)
- [Physics-Informed Deep Neural Operator Networks](https://mlsanity.com/paper/2207.05748/) (2022)
- [A physics-informed variational DeepONet for predicting the crack path in brittle materials](https://mlsanity.com/paper/2108.06905/) (2021)
- [Tackling the Curse of Dimensionality with Physics-Informed Neural Networks](https://mlsanity.com/paper/2307.12306/) (2023)

## Built on by (most cited first, 28 of 28)

- [Oscillatory State-Space Models](https://mlsanity.com/paper/2410.03943/) (2024)
- [Mamba Neural Operator: Who Wins? Transformers vs. State-Space Models for PDEs](https://mlsanity.com/paper/2410.02113/) (2024)
- [Recurrent Neural Operators: Stable Long-Term PDE Prediction](https://mlsanity.com/paper/2505.20721/) (2025)
- [Comba: Improving Bilinear RNNs with Closed-loop Control](https://mlsanity.com/paper/2506.02475/) (2025)
- [Sub-Sequential Physics-Informed Learning with State Space Model](https://mlsanity.com/paper/2502.00318/) (2025)
- [Representation Meets Optimization: Training PINNs and PIKANs for Gray-Box Discovery in Systems Pharmacology](https://mlsanity.com/paper/2504.07379/) (2025)
- [Latent Mamba Operator for Partial Differential Equations](https://mlsanity.com/paper/2505.19105/) (2025)
- [Learning to Dissipate Energy in Oscillatory State-Space Models](https://mlsanity.com/paper/2505.12171/) (2025)
- [Temporal Neural Operator for Modeling Time-Dependent Physical Phenomena](https://mlsanity.com/paper/2504.20249/) (2025)
- [Block-Biased Mamba for Long-Range Sequence Processing](https://mlsanity.com/paper/2505.09022/) (2025)
- [CMINNs: Compartment Model Informed Neural Networks -- Unlocking Drug Dynamics](https://mlsanity.com/paper/2409.12998/) (2024)
- [Learning PDE Solvers with Physics and Data: A Unifying View of Physics-Informed Neural Networks and Neural Operators](https://mlsanity.com/paper/2601.14517/) (2026)
- [Physics-Informed Machine Learning in Biomedical Science and Engineering](https://mlsanity.com/paper/2510.05433/) (2025)
- [Adversarial Vulnerabilities in Neural Operator Digital Twins: Gradient-Free Attacks on Nuclear Thermal-Hydraulic Surrogates](https://mlsanity.com/paper/2603.22525/) (2026)
- [Automatic selection of the best neural architecture for time series forecasting](https://mlsanity.com/paper/2501.12215/) (2025)
- [GeoMaNO: Geometric Mamba Neural Operator for Partial Differential Equations](https://mlsanity.com/paper/2505.12020/) (2025)
- [Merging Memory and Space: A State Space Neural Operator](https://mlsanity.com/paper/2507.23428/) (2025)
- [AMORE: Adaptive Multi-Output Operator Network for Stiff Chemical Kinetics](https://mlsanity.com/paper/2510.12999/) (2025)
- [Geometric Neural Operators via Lie Group-Constrained Latent Dynamics](https://mlsanity.com/paper/2602.16209/) (2026)
- [FMaMIL: Frequency-Driven Mamba Multi-Instance Learning for Weakly Supervised Lesion Segmentation in Medical Images](https://mlsanity.com/paper/2506.07652/) (2025)
- [Neuronal Group Communication for Efficient Neural representation](https://mlsanity.com/paper/2510.16851/) (2025)
- [Adaptive Mamba Neural Operators](https://mlsanity.com/paper/2607.18043/) (2026)
- [Variational autoencoders with latent high-dimensional steady geometric flows for dynamics](https://mlsanity.com/paper/2410.10137/) (2024)
- [Drivetrain simulation using variational autoencoders](https://mlsanity.com/paper/2501.17653/) (2025)
- [Observability conditions for neural state-space models with eigenvalues and their roots of unity](https://mlsanity.com/paper/2504.15758/) (2025)
- [ASecond-Order SpikingSSM for Wearables](https://mlsanity.com/paper/2510.14386/) (2025)
- [Kinetic-Mamba: Mamba-Assisted Predictions of Stiff Chemical Kinetics](https://mlsanity.com/paper/2512.14471/) (2025)
- [Time-Delayed Transformers for Data-Driven Modeling of Low-Dimensional Dynamics](https://mlsanity.com/paper/2602.08478/) (2026)

## Abstract (arXiv)

Physics-informed machine learning (PIML) has emerged as a promising alternative to classical methods for predicting dynamical systems, offering faster and more generalizable solutions. However, existing models, including recurrent neural networks (RNNs), transformers, and neural operators, face challenges such as long-time integration, long-range dependencies, chaotic dynamics, and extrapolation, to name a few. To this end, this paper introduces state-space models implemented in Mamba for accurate and efficient dynamical system operator learning. Mamba addresses the limitations of existing architectures by dynamically capturing long-range dependencies and enhancing computational efficiency through reparameterization techniques. To extensively test Mamba and compare against another 11 baselines, we introduce several strict extrapolation testbeds that go beyond the standard interpolation benchmarks. We demonstrate Mamba's superior performance in both interpolation and challenging extrapolation tasks. Mamba consistently ranks among the top models while maintaining the lowest computational cost and exceptional extrapolation capabilities. Moreover, we demonstrate the good performance of Mamba for a real-world application in quantitative systems pharmacology for assessing the efficacy of drugs in tumor growth under limited data scenarios. Taken together, our findings highlight Mamba's potential as a powerful tool for advancing scientific machine learning in dynamical systems modeling. (The code will be available at https://github.com/zheyuanhu01/State_Space_Model_Neural_Operator upon acceptance.)