mlsanityData to 7 Oct 2026

SPIN: Shadow Predictive Indexer for Sparse Attention

arXiv 2610.09025 ↗First posted 6 Oct 2026cs.LGcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Indexer-based sparse attention reduces the cost of core attention by passing only a fixed, small number of important tokens to it. However, the indexer must still score the entire KV cache at every decoding step. This scoring overhead becomes a major bottleneck as the context length grows. We propose SPIN (Shadow Predictive Indexer) to reduce this indexer overhead. SPIN uses lightweight, history-based prediction to identify important KV blocks, avoiding the need to score the full KV cache at every decoding step. SPIN treats KV blocks and speculative decoding as first-class design and implementation considerations. Across extensive evaluations on long-context and agentic benchmarks, SPIN achieves 30-40% sparsity while preserving task quality. In end-to-end vLLM serving, SPIN improves output throughput by up to 14.9% and reduces median inter-token latency by up to 13.2%.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 23 of its 30 references that are in the map. First posted on arXiv 6 Oct 2026.

Builds on

23 of 30 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026Guess-Verify-Refine: Data-Aware Top-K for Sparse-Attention Decoding on Blackwell via Temporal Correlation24 Apr
  2. 2026Why Attention Patterns Exist: A Unifying Temporal Perspective Analysis29 Jan
  3. 2025AttentionPredictor: Temporal Patterns Matter for KV Cache Compression6 Feb
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep
  5. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 18

  1. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  2. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  3. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  4. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  5. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
  6. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
  7. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
  8. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 367
  9. 2024Round and Round We Go! What makes Rotary Positional Encodings useful?cited by 111
  10. 2021SummScreen: A Dataset for Abstractive Screenplay Summarizationcited by 80
  11. 2026MiniMax Sparse Attentioncited by 34
  12. 2026IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reusecited by 27
6 more
  1. 2026HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attentioncited by 10
  2. 2026SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decodingcited by 7
  3. 2026MISA: Mixture of Indexer Sparse Attention for Long-Context LLM Inferencecited by 5
  4. 2026Demystifying the Slash Pattern in Attention: The Role of RoPEcited by 2
  5. 2026Predict, Reuse, and Repair: Accelerating Dynamic Sparse Attention for Long-Context LLM Decodingcited by 2
  6. 2026PIVOT: Efficient Query-Group Indexing for Token-Level Sparse Attentioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving
  2. 2026You Only Index Once: Cross-Layer Sparse Attention with Shared Routing
  3. 2026Sparse Attention as a Range Searching Problem: Towards an Inference-Efficient Index for KV Cache
  4. 2026Self-Indexing Attention for Compression-Compatible Sparse Long-Context LLM Inference
  5. 2026HySparse2: Hybrid Sparse Attention with Two-Level KV Sharing
  6. 2025SpecAttn: Speculating Sparse Attention
  7. 2026Augmenting Attention with Exponentially Decaying Memory Improves Query-Aware KV Sparsity
  8. 2026Scaling Attention via Feature Sparsity