mlsanityData to 14 Oct 2026

Kascade: A Practical Sparse Attention Method for Long-Context LLM Inference

arXiv 2512.16391 ↗First posted 18 Dec 2025cs.LG cs.AI cs.DCcited by 10

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Attention is the dominant source of latency during long-context LLM inference, an increasingly popular workload with reasoning models and RAG. We propose Kascade, a training-free sparse attention method that leverages known observations such as 1) post-softmax attention is intrinsically sparse, and 2) the identity of high-weight keys is stable across nearby layers. Kascade computes exact Top-k indices in a small set of anchor layers, then reuses those indices in intermediate reuse layers. The anchor layers are selected algorithmically, via a dynamic-programming objective that maximizes cross-layer similarity over a development set, allowing easy deployment across models. The method incorporates efficient implementation constraints (e.g. tile-level operations), across both prefill and decode attention. The Top-k selection and reuse in Kascade is head-aware and we show in our experiments that this is critical for high accuracy. Kascade achieves up to 4.1x speedup in decode attention and 2.2x speedup in prefill attention over FlashAttention-3 baseline on H100 GPUs while closely matching dense attention accuracy on long-context benchmarks such as LongBench and AIME-24.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 28 of its 32 references that are in the map. First posted on arXiv 18 Dec 2025.

Builds on

28 of 32 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025gpt-oss-120b & gpt-oss-20b Model Card8 Aug
  2. 2025SeerAttention-R: Sparse Attention Adaptation for Long Reasoning10 Jun

Other references · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  5. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  6. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  7. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  8. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  9. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  10. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 519
  11. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  12. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 383
14 more
  1. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
  2. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 223
  3. 2024SeerAttention: Learning Intrinsic Sparse Attention in Your LLMscited by 124
  4. 2023SparQ Attention: Bandwidth-Efficient LLM Inferencecited by 119
  5. 2024Loki: Low-rank Keys for Efficient Sparse Attentioncited by 89
  6. 2024What Matters in Transformers? Not All Attention is Neededcited by 84
  7. 2024TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Textcited by 63
  8. 2025TileLang: A Composable Tiled Programming Model for AI Systemscited by 43
  9. 2025LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attentioncited by 41
  10. 2024Block-Attention for Efficient Prefillingcited by 37
  11. 2019Sharing Attention Weights for Fast Transformercited by 34
  12. 2021Leveraging redundancy in attention with Reuse Transformerscited by 33
  13. 2024TidalDecode: Fast and Accurate LLM Decoding with Position Persistent Sparse Attentioncited by 29
  14. 2021LazyFormer: Self Attention with Lazy Updatecited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 10.

10 papers.

  1. 2026IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reusecited by 31
  2. 2026HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharingcited by 10
  3. 2026You Only Index Once: Cross-Layer Sparse Attention with Shared Routingcited by 2
  4. 2026LongCat Sparse Attention: Taming the Lightning via Streaming-aware Hierarchical Cross-Layer Indexingcited by 2
  5. 2026LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decodingcited by 1
  6. 2026Recall Before You Rank: Similarity-Guided Top-K Reuse for Efficient Long-Context Attentioncited by 1
  7. 2026Autonomy-of-Heads: Data-Free Sparse Attention from Frozen Query-Key Geometrycited by 1
  8. 2026Attention Routing Stabilizes Early: Working-Set Inference for Recurrent Language Modelscited by 1
  9. 2026The Sirens' Song: When Proximal Background Context Overshadows Distant Evidencecited by 0
  10. 2026The Evolution of Attention in Large Language Models: Mechanisms, Trade-offs, and Emerging Trendscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference
  2. 2025Long-Context Modeling with Dynamic Hierarchical Sparse Attention for Memory-Constrained LLM Inference
  3. 2025Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs
  4. 2025SALE : Low-bit Estimation for Efficient Sparse Attention in Long-context LLM Prefilling
  5. 2026CSAttention: Centroid-Scoring Attention for Accelerating LLM Inference
  6. 2026Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference
  7. 2026AsyncTLS: Efficient Generative LLM Inference with Asynchronous Two-level Sparse Attention
  8. 2026LycheeDecode: Accelerating Long-Context LLM Inference via Hybrid-Head Sparse Decoding