# Analysis of Long Range Dependency Understanding in State Space Models

arXiv: https://arxiv.org/abs/2601.13048 · HTML: https://mlsanity.com/paper/2601.13048/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 18 references that are in the map.
- First posted on arXiv 19 Jan 2026.

## Builds on (most cited first, 17 of 17)

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021)
- [On the Parameterization and Initialization of Diagonal State Space Models](https://mlsanity.com/paper/2206.11893/) (2022)
- [Deep Learning based Vulnerability Detection: Are We There Yet?](https://mlsanity.com/paper/2009.07235/) (2020)
- [Finite Depth and Width Corrections to the Neural Tangent Kernel](https://mlsanity.com/paper/1909.05989/) (2019)
- [S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spaces](https://mlsanity.com/paper/2210.06583/) (2022)
- [StableSSM: Alleviating the Curse of Memory in State-space Models through Stable Reparameterization](https://mlsanity.com/paper/2311.14495/) (2023)
- [Spectral Leakage and Rethinking the Kernel Size in CNNs](https://mlsanity.com/paper/2101.10143/) (2021)
- [From Generalization Analysis to Optimization Designs for State Space Models](https://mlsanity.com/paper/2405.02670/) (2024)
- [Autocorrelation Matters: Understanding the Role of Initialization Schemes for State Space Models](https://mlsanity.com/paper/2411.19455/) (2024)
- [SMR: State Memory Replay for Long Sequence Modeling](https://mlsanity.com/paper/2405.17534/) (2024)
- [State Space Models are Provably Comparable to Transformers in Dynamic Token Selection](https://mlsanity.com/paper/2405.19036/) (2024)
- [CodeSSM: Towards State Space Models for Code Understanding](https://mlsanity.com/paper/2505.01475/) (2025)

## Built on by (most cited first, 1 of 1)

- [Towards Understanding What State Space Models Learn About Code](https://mlsanity.com/paper/2602.06774/) (2026)

## Abstract (arXiv)

Although state-space models (SSMs) have demonstrated strong performance on long-sequence benchmarks, most research has emphasized predictive accuracy rather than interpretability. In this work, we present the first systematic kernel interpretability study of the diagonalized state-space model (S4D) trained on a real-world task (vulnerability detection in source code). Through time and frequency domain analysis of the S4D kernel, we show that the long-range modeling capability of S4D varies significantly under different model architectures, affecting model performance. For instance, we show that the depending on the architecture, S4D kernel can behave as low-pass, band-pass or high-pass filter. The insights from our analysis can guide future work in designing better S4D-based models.