mlsanityData to 14 Oct 2026

Technologies on Effectiveness and Efficiency: A Survey of State Spaces Models

arXiv 2503.11224 ↗First posted 14 Mar 2025cs.LG cs.AI cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

State Space Models (SSMs) have emerged as a promising alternative to the popular transformer-based models and have been increasingly gaining attention. Compared to transformers, SSMs excel at tasks with sequential data or longer contexts, demonstrating comparable performances with significant efficiency gains. In this survey, we provide a coherent and systematic overview for SSMs, including their theoretical motivations, mathematical formulations, comparison with existing model classes, and various applications. We divide the SSM series into three main sections, providing a detailed introduction to the original SSM, the structured SSM represented by S4, and the selective SSM typified by Mamba. We put an emphasis on technicality, and highlight the various key techniques introduced to address the effectiveness and efficiency of SSMs. We hope this manuscript serves as an introduction for researchers to explore the theoretical foundations of SSMs.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 85 of its 127 references that are in the map. First posted on arXiv 14 Mar 2025.

Builds on

85 of 127 references.

References · 85

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  3. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  4. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  5. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 700
  6. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 605
  7. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 424
  8. 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 401
  9. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 355
  10. 2024MambaIR: A Simple Baseline for Image Restoration with State-Space Modelcited by 352
  11. 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 328
  12. 2024SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentationcited by 266
38 more
  1. 2022Long Range Language Modeling via Gated State Spacescited by 265
  2. 2024LocalMamba: Visual State Space Model with Windowed Selective Scancited by 236
  3. 2024Repeat After Me: Transformers are Better than State Space Models at Copyingcited by 216
  4. 2023HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolutioncited by 194
  5. 2024Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modelingcited by 174
  6. 2024PointMamba: A Simple State Space Model for Point Cloud Analysiscited by 167
  7. 2022Liquid Structural State-Space Modelscited by 136
  8. 2024Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spacescited by 134
  9. 2024The Mamba in the Llama: Distilling and Accelerating Hybrid Modelscited by 122
  10. 2024Is Mamba Effective for Time Series Forecasting?cited by 114
  11. 2024Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modelingcited by 113
  12. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understandingcited by 108
  13. 2022Long Movie Clip Classification with State-Space Video Modelscited by 106
  14. 2024Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Taskscited by 105
  15. 2024DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmarkcited by 92
  16. 2024Graph Mamba: Towards Learning on Graphs with State Space Modelscited by 86
  17. 2024SiMBA: Simplified Mamba-Based Architecture for Vision and Multivariate Time seriescited by 82
  18. 2024Theoretical Foundations of Deep Selective State-Space Modelscited by 74
  19. 2024Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Modelscited by 67
  20. 2024TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecastingcited by 64
  21. 2024An Investigation of Incorporating Mamba for Speech Enhancementcited by 62
  22. 2024MoE-Mamba: Efficient Selective State Space Models with Mixture of Expertscited by 60
  23. 2024Mamba in Speech: Towards an Alternative to Self-Attentioncited by 60
  24. 2024MambaByte: Token-free Selective State Space Modelcited by 59
  25. 2024Mamba3D: Enhancing Local Features for 3D Point Cloud Analysis via State Space Modelcited by 57
  26. 2024Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstructioncited by 56
  27. 2024RainMamba: Enhanced Locality Learning with State Space Models for Video Derainingcited by 56
  28. 2023Effectively Modeling Time Series with Simple Discrete State Spacescited by 54
  29. 2022Pretraining Without Attentioncited by 51
  30. 2022Efficient Movie Scene Detection using State-Space Transformerscited by 51
  31. 2023GateLoop: Fully Data-Controlled Linear Recurrence for Sequence Modelingcited by 46
  32. 2024nnMamba: 3D Biomedical Image Segmentation, Classification and Landmark Detection with State Space Modelcited by 45
  33. 2024Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separationcited by 45
  34. 2024Hydra: Bidirectional State Space Models Through Generalized Matrix Mixerscited by 44
  35. 2024MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selectioncited by 40
  36. 2022Efficient Long Sequence Modeling via State Space Augmented Transformercited by 37
  37. 2024BlackMamba: Mixture of Experts for State-Space Modelscited by 37
  38. 2024SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space Modelcited by 37

50 most cited of 85 All 85 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Reasoning Beyond Limits: Advances and Open Problems for LLMscited by 16
  2. 2025StateX: Enhancing RNN Recall via Post-training State Expansioncited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025On Pruning State-Space LLMs
  2. 2024State Space Model for New-Generation Network Alternative to Transformers: A Survey
  3. 2025State-offset Tuning: State-based Parameter-Efficient Fine-Tuning for State Space Models
  4. 2024The Expressive Capacity of State Space Models: A Formal Language Perspective
  5. 2026Benchmarking the Computational and Representational Efficiency of State Space Models against Transformers on Long-Context Dyadic Sessions
  6. 2024The Illusion of State in State-Space Models
  7. 2025Memory Determines Learning Direction: A Theory of Gradient-Based Optimization in State Space Models
  8. 2026Attention to Mamba: A Recipe for Cross-Architecture Distillation