mlsanityData to 14 Oct 2026

SuPER-Mamba: Surprisal Prioritization for Extended-Context Routing in Mamba

arXiv 2610.16636 ↗First posted 12 Oct 2026cs.CL cs.AIcited by 0

On the map: State-space sequence models (S4, Mamba) → Core SSM architectures for language

Abstract (arXiv)

State space models provide efficient linear-complexity processing for long sequences, but their performance frequently degrades as context length grows. We propose SuPER-Mamba, a training-free framework that uses Surprisal as a model-intrinsic signal for long-range state routing in pretrained Mamba models. SuPER-Mamba computes position-wise Surprisal from next-token predictions, aggregates neighboring responses through Local Pooling, and uses the resulting scores to prioritize Global-Channel updates without query-dependent scoring or additional training. We evaluate SuPER-Mamba on three state-space models across the LongBench-E benchmark. The results show that SuPER-Mamba improves the average score from 8.06 to 15.69, corresponding to a 94.7% relative gain over the Base Models and a 13.1% improvement over the state-of-the-art prior baseline. Ablation and length-wise analyses further support the effectiveness of the proposed routing framework.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 25 of its 36 references that are in the map. First posted on arXiv 12 Oct 2026.

Builds on

25 of 36 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026Mamba with Hierarchical Memory: Solving Representation Bottleneck in Long Sequence Modeling3 Aug
  2. 2026Mamba-3: Improved Sequence Modeling using State Space Principles16 Mar
  3. 2025Characterizing Mamba's Selective Memory using Auto-Encoders17 Dec
  4. 2025Recall with Reasoning: Chain-of-Thought Distillation for Mamba's Long-Context Memory and Extrapolation6 May
  5. 2025LongMamba: Enhancing Mamba's Long Context Capabilities via Training-Free Receptive Field Enlargement22 Apr
  6. 2025LaTIM: Measuring Latent Token-to-Token Interactions in Mamba Models21 Feb
  7. 2024Revealing and Mitigating the Local Pattern Shortcuts of Mamba21 Oct
  8. 2024ReMamba: Equip Mamba with Effective Long-Sequence Modeling28 Aug
  9. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality31 May
  10. 2024The Hidden Attention of Mamba Models3 Mar
  11. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spaces1 Dec

Other references · 14

  1. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  2. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  3. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 474
  4. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 453
  5. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 439
  6. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
  7. 2025ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inferencecited by 60
  8. 2024Perception Compressor: A Training-Free Prompt Compression Framework in Long Context Scenarioscited by 23
  9. 2025DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft Tokenscited by 22
  10. 2025Efficient Context Selection for Long-Context QA: No Tuning, No Iteration, Just Adaptive-kcited by 21
  11. 2025Understanding and Improving Information Preservation in Prompt Compression for LLMscited by 15
  12. 2025Hardware-aligned Hierarchical Sparse Attention for Efficient Long-term Memory Accesscited by 9
2 more
  1. 2025AttentionPredictor: Temporal Patterns Matter for KV Cache Compressioncited by 7
  2. 2024QUITO-X: A New Perspective on Context Compression from the Information Bottleneck Theorycited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection
  2. 2026An Exact Instrument for State Usage in Selective State-Space Models, and the Input-Driven Migration It Reveals
  3. 2025Block-Biased Mamba for Long-Range Sequence Processing
  4. 2025Efficient Unstructured Pruning of Mamba State-Space Models for Resource-Constrained Environments
  5. 2024Rethinking Token Reduction for State Space Models
  6. 2024Bi-Mamba: Towards Accurate 1-Bit State Space Models
  7. 2025Lookahead Routing for Large Language Models
  8. 2026When Does Content-Based Routing Work? Representation Requirements for Selective Attention in Hybrid Sequence Models