# PerfMamba: Performance Analysis and Pruning of Selective State Space Models

arXiv: https://arxiv.org/abs/2511.22849 · HTML: https://mlsanity.com/paper/2511.22849/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 24 of its 32 references that are in the map.
- First posted on arXiv 28 Nov 2025.

## Placed in

- State-space sequence models (S4, Mamba) / [Core SSM architectures for language](https://mlsanity.com/ssm/core-ssm-architectures-for-language/)

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021)
- [A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations](https://mlsanity.com/paper/2308.06767/) (2023)
- [An Empirical Study of Mamba-based Language Models](https://mlsanity.com/paper/2406.07887/) (2024)
- [Structured Pruning Learns Compact and Accurate Models](https://mlsanity.com/paper/2204.00408/) (2022)
- [The Efficiency Misnomer](https://mlsanity.com/paper/2110.12894/) (2021)
- [The Hidden Attention of Mamba Models](https://mlsanity.com/paper/2403.01590/) (2024)
- [NormFormer: Improved Transformer Pretraining with Extra Normalization](https://mlsanity.com/paper/2110.09456/) (2021)
- [A Survey on Visual Mamba](https://mlsanity.com/paper/2404.15956/) (2024)
- [Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes](https://mlsanity.com/paper/2402.05406/) (2024)
- [When to Prune? A Policy towards Early Structural Pruning](https://mlsanity.com/paper/2110.12007/) (2021)
- [Improving Machine Reading Comprehension with General Reading Strategies](https://mlsanity.com/paper/1810.13441/) (2018)
- [Pruning Foundation Models for High Accuracy without Retraining](https://mlsanity.com/paper/2410.15567/) (2024)
- [Rethinking Token Reduction for State Space Models](https://mlsanity.com/paper/2410.14725/) (2024)
- [MARCA: Mamba Accelerator with ReConfigurable Architecture](https://mlsanity.com/paper/2409.11440/) (2024)
- [Mamba-Spike: Enhancing the Mamba Architecture with a Spiking Front-End for Efficient Temporal Data Processing](https://mlsanity.com/paper/2408.11823/) (2024)
- [Mamba-Shedder: Post-Transformer Compression for Efficient Selective Structured State Space Models](https://mlsanity.com/paper/2501.17088/) (2025)
- [Deep Gate Recurrent Neural Network](https://mlsanity.com/paper/1604.02910/) (2016)

## Built on by (most cited first, 1 of 1)

- [An Exact Instrument for State Usage in Selective State-Space Models, and the Input-Driven Migration It Reveals](https://mlsanity.com/paper/2607.11796/) (2026)

## Abstract (arXiv)

Recent advances in sequence modeling have introduced selective SSMs as promising alternatives to Transformer architectures, offering theoretical computational efficiency and sequence processing advantages. A comprehensive understanding of selective SSMs in runtime behavior, resource utilization patterns, and scaling characteristics still remains unexplored, thus obstructing their optimal deployment and further architectural improvements. This paper presents a thorough empirical study of Mamba-1 and Mamba-2, systematically profiled for performance to assess the design principles that contribute to their efficiency in state-space modeling. A detailed analysis of computation patterns, memory access, I/O characteristics, and scaling properties was performed for sequence lengths ranging from 64 to 16384 tokens. Our findings show that the SSM component, a central part of the selective SSM architecture, demands a significant portion of computational resources compared to other components in the Mamba block. Based on these insights, we propose a pruning technique that selectively removes low-activity states within the SSM component, achieving measurable throughput and memory gains while maintaining accuracy within a moderate pruning regime. This approach results in performance improvements across varying sequence lengths, achieving a 1.14x speedup and reducing memory usage by 11.50%. These results offer valuable guidance for designing more efficient SSM architectures that can be applied to a wide range of real-world applications.