# SuPER-Mamba: Surprisal Prioritization for Extended-Context Routing in Mamba

arXiv: https://arxiv.org/abs/2610.16636 · HTML: https://mlsanity.com/paper/2610.16636/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 12 Oct 2026.

## Placed in

- State-space sequence models (S4, Mamba) / [Core SSM architectures for language](https://mlsanity.com/ssm/core-ssm-architectures-for-language/)

## Builds on (most cited first, 25 of 25)

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [SnapKV: LLM Knows What You are Looking for Before Generation](https://mlsanity.com/paper/2404.14469/) (2024)
- [Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs](https://mlsanity.com/paper/2310.01801/) (2023)
- [LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression](https://mlsanity.com/paper/2310.06839/) (2023)
- [Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time](https://mlsanity.com/paper/2305.17118/) (2023)
- [PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling](https://mlsanity.com/paper/2406.02069/) (2024)
- [The Hidden Attention of Mamba Models](https://mlsanity.com/paper/2403.01590/) (2024)
- [Mamba-3: Improved Sequence Modeling using State Space Principles](https://mlsanity.com/paper/2603.15569/) (2026)
- [ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference](https://mlsanity.com/paper/2502.00299/) (2025)
- [LongMamba: Enhancing Mamba's Long Context Capabilities via Training-Free Receptive Field Enlargement](https://mlsanity.com/paper/2504.16053/) (2025)
- [Perception Compressor: A Training-Free Prompt Compression Framework in Long Context Scenarios](https://mlsanity.com/paper/2409.19272/) (2024)
- [DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft Tokens](https://mlsanity.com/paper/2502.11493/) (2025)
- [Efficient Context Selection for Long-Context QA: No Tuning, No Iteration, Just Adaptive-k](https://mlsanity.com/paper/2506.08479/) (2025)
- [Understanding and Improving Information Preservation in Prompt Compression for LLMs](https://mlsanity.com/paper/2503.19114/) (2025)
- [ReMamba: Equip Mamba with Effective Long-Sequence Modeling](https://mlsanity.com/paper/2408.15496/) (2024)
- [Hardware-aligned Hierarchical Sparse Attention for Efficient Long-term Memory Access](https://mlsanity.com/paper/2504.16795/) (2025)
- [AttentionPredictor: Temporal Patterns Matter for KV Cache Compression](https://mlsanity.com/paper/2502.04077/) (2025)
- [LaTIM: Measuring Latent Token-to-Token Interactions in Mamba Models](https://mlsanity.com/paper/2502.15612/) (2025)
- [QUITO-X: A New Perspective on Context Compression from the Information Bottleneck Theory](https://mlsanity.com/paper/2408.10497/) (2024)
- [Characterizing Mamba's Selective Memory using Auto-Encoders](https://mlsanity.com/paper/2512.15653/) (2025)
- [Revealing and Mitigating the Local Pattern Shortcuts of Mamba](https://mlsanity.com/paper/2410.15678/) (2024)
- [Recall with Reasoning: Chain-of-Thought Distillation for Mamba's Long-Context Memory and Extrapolation](https://mlsanity.com/paper/2505.03320/) (2025)
- [Mamba with Hierarchical Memory: Solving Representation Bottleneck in Long Sequence Modeling](https://mlsanity.com/paper/2608.02347/) (2026)

## Abstract (arXiv)

State space models provide efficient linear-complexity processing for long sequences, but their performance frequently degrades as context length grows. We propose SuPER-Mamba, a training-free framework that uses Surprisal as a model-intrinsic signal for long-range state routing in pretrained Mamba models. SuPER-Mamba computes position-wise Surprisal from next-token predictions, aggregates neighboring responses through Local Pooling, and uses the resulting scores to prioritize Global-Channel updates without query-dependent scoring or additional training. We evaluate SuPER-Mamba on three state-space models across the LongBench-E benchmark. The results show that SuPER-Mamba improves the average score from 8.06 to 15.69, corresponding to a 94.7% relative gain over the Base Models and a 13.1% improvement over the state-of-the-art prior baseline. Ablation and length-wise analyses further support the effectiveness of the proposed routing framework.