An Exact Instrument for State Usage in Selective State-Space Models, and the Input-Driven Migration It Reveals
On the map: State-space sequence models (S4, Mamba) → Core SSM architectures for language
Abstract (arXiv)
Selective state-space models such as Mamba route information through a bank of first-order modes whose input coupling is set by a learned selection mechanism. We give an exact instrument for measuring how a trained model uses these modes. Because the state matrix is diagonal, each channel's output decomposes exactly into per-mode contributions, and a per-(layer, channel, window) Gram tensor yields the exact output error of dropping any subset of modes, offline, at any budget. Validated against the reference implementation to a relative error of 2.3×10-7 on the Mamba-1 family where it is exact, the instrument predicts a layer's deployed pruning error to a median relative deviation of 5×10-7 over 4,464 configurations, its floor set by the reconstruction. Applying the instrument across the Mamba-1 family (130M--2.8B), the deployed 7B Falcon-Mamba, and Mamba-2, we find that trained models re-allocate their state space with the input: which modes carry the signal migrates across contexts, and at the most affected layers a per-input oracle roughly halves the output error of a fixed mode set. Frozen-signal counterfactuals attribute the migration primarily to the input-dependent write map Bt; the timestep usually identified with selectivity carries almost none of it. Input-scheduled mode pruning on this measurement outperforms static, Hankel-based, and layer-adaptive rankings at every scale from 130M to the deployed 7B Falcon-Mamba, and at half the state budget it matches the unpruned model. Because the scheduler reads each window's mode usage from a first pass, this demonstrates realizable headroom; we claim no deployed compute or memory saving.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 12 of its 16 references that are in the map. First posted on arXiv 13 Jul 2026.
Builds on
12 of 16 references.
Closest ancestors · 8
Same direction, newest first
- 2025Characterizing Mamba's Selective Memory using Auto-Encoders17 Dec
- 2025PerfMamba: Performance Analysis and Pruning of Selective State Space Models28 Nov
- 2025Efficient Unstructured Pruning of Mamba State-Space Models for Resource-Constrained Environments13 May
- 2024Falcon Mamba: The First Competitive Attention-free 7B Language Model7 Oct
- 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality31 May
- 2024The Hidden Attention of Mamba Models3 Mar
- 2024Model Compression Method for S4 with Diagonal State Space Layers using Balanced Truncation25 Feb
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spaces1 Dec
Other references · 4
- 2024Layer-Adaptive State Pruning for Deep State Space Modelscited by 10
- 2024Model order reduction of deep structured state-space models: A system-theoretic approachcited by 6
- 2025Hankel Singular Value Regularization for Highly Compressible State Space Modelscited by 2
- 2026GHOST: Unmasking Phantom States in Mamba2 via Grouped Hidden-state Output-aware Selection & Truncationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Demystifying the Token Dynamics of Deep Selective State Space Models
- 2025Rethinking Selectivity in State Space Models: A Minimal Predictive Sufficiency Approach
- 2025Block-Biased Mamba for Long-Range Sequence Processing
- 2024Theoretical Foundations of Deep Selective State-Space Models
- 2026A Theoretical Analysis of Mamba's Training Dynamics: Filtering Relevant Features for Generalization in State Space Models
- 2026Mamba-3: Improved Sequence Modeling using State Space Principles
- 2024Rethinking Token Reduction for State Space Models
- 2026TIDES: Implicit Time-Awareness in Selective State Space Models