# SPIN: Shadow Predictive Indexer for Sparse Attention

arXiv: https://arxiv.org/abs/2610.09025 · HTML: https://mlsanity.com/paper/2610.09025/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving.
- Builds on 21 of its 28 references that are in the map.
- First posted on arXiv 6 Oct 2026.

## Placed in

- Efficient LLM inference and serving / [KV cache optimization](https://mlsanity.com/inference/kv-cache-optimization/)

## Builds on (most cited first, 21 of 21)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models](https://mlsanity.com/paper/2306.14048/) (2023)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://mlsanity.com/paper/2506.07982/) (2025)
- [LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks](https://mlsanity.com/paper/2412.15204/) (2024)
- [MiniMax Sparse Attention](https://mlsanity.com/paper/2606.13392/) (2026)
- [IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse](https://mlsanity.com/paper/2603.12201/) (2026)
- [HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention](https://mlsanity.com/paper/2603.28458/) (2026)
- [SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding](https://mlsanity.com/paper/2604.09557/) (2026)
- [AttentionPredictor: Temporal Patterns Matter for KV Cache Compression](https://mlsanity.com/paper/2502.04077/) (2025)
- [MISA: Mixture of Indexer Sparse Attention for Long-Context LLM Inference](https://mlsanity.com/paper/2605.07363/) (2026)
- [Why Attention Patterns Exist: A Unifying Temporal Perspective Analysis](https://mlsanity.com/paper/2601.21709/) (2026)
- [Demystifying the Slash Pattern in Attention: The Role of RoPE](https://mlsanity.com/paper/2601.08297/) (2026)
- [Guess-Verify-Refine: Data-Aware Top-K for Sparse-Attention Decoding on Blackwell via Temporal Correlation](https://mlsanity.com/paper/2604.22312/) (2026)
- [PIVOT: Efficient Query-Group Indexing for Token-Level Sparse Attention](https://mlsanity.com/paper/2607.24593/) (2026)
- [Predict, Reuse, and Repair: Accelerating Dynamic Sparse Attention for Long-Context LLM Decoding](https://mlsanity.com/paper/2606.30389/) (2026)

## Abstract (arXiv)

Indexer-based sparse attention reduces the cost of core attention by passing only a fixed, small number of important tokens to it. However, the indexer must still score the entire KV cache at every decoding step. This scoring overhead becomes a major bottleneck as the context length grows. We propose SPIN (Shadow Predictive Indexer) to reduce this indexer overhead. SPIN uses lightweight, history-based prediction to identify important KV blocks, avoiding the need to score the full KV cache at every decoding step. SPIN treats KV blocks and speculative decoding as first-class design and implementation considerations. Across extensive evaluations on long-context and agentic benchmarks, SPIN achieves 30-40% sparsity while preserving task quality. In end-to-end vLLM serving, SPIN improves output throughput by up to 14.9% and reduces median inter-token latency by up to 13.2%.