mlsanityData to 7 Oct 2026

Salca: A Sparsity-Aware Hardware Accelerator for Efficient Long-Context Attention Decoding

arXiv 2604.24820 ↗First posted 27 Apr 2026cs.AR cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Long contexts improve capabilities of large language models but pose serious hardware challenges: compute and memory footprints grow linearly with sequence length. Particularly, the decoding phase continuously accesses massive KV cache, dramatically increasing bandwidth and computing pressure. Existing accelerators are primarily designed and evaluated for short contexts. They suffer from significant performance degradation when processing long contexts. To bridge this gap, we identify the major bottleneck and present a hardware accelerator for long context attention decoding via hardware-software co-design. On the software side, we propose dual-compression dynamic sparse attention. It combines ultra-low-precision quantization with feature sparsity to minimize prediction overhead. A hardware-friendly approximate Top-K selection further reduces filter complexity from O(n log k) to O(n). On the hardware side, we deeply optimize compute and memory access to tackle bottlenecks from intricate interplay between sparse attention and long contexts, and establish a performance model to derive the optimal co-design scheme. The resulting hardware adopts a fully pipelined parallel architecture and achieves O(n) efficiency even for long sequences. Experiments show that our design delivers 3.82× speedup and 74.19× energy efficiency over A100. Compared to SOTA accelerators, this is the first ASIC accelerator that efficiently supports long context inference, with at least 3.5× higher throughput and 2.08× better energy efficiency.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 43 of its 67 references that are in the map. First posted on arXiv 27 Apr 2026.

Builds on

43 of 67 references.

References · 43

  1. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  2. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  3. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  4. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  5. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  6. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  7. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  8. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 487
  9. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 420
  10. 20218-bit Optimizers via Block-wise Quantizationcited by 395
  11. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 391
  12. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 375
31 more
  1. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 255
  2. 2020Sparse Sinkhorn Attentioncited by 250
  3. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementcited by 242
  4. 2020SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningcited by 228
  5. 2025MoBA: Mixture of Block Attention for Long-Context LLMscited by 188
  6. 2023SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefillscited by 187
  7. 2024SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Accelerationcited by 158
  8. 2018Online normalizer calculation for softmaxcited by 154
  9. 2019Adaptively Sparse Transformerscited by 154
  10. 2024SeerAttention: Learning Intrinsic Sparse Attention in Your LLMscited by 121
  11. 2023SparQ Attention: Bandwidth-Efficient LLM Inferencecited by 119
  12. 2020A3: Accelerating Attention Mechanisms in Neural Networks with Approximationcited by 97
  13. 2024PQCache: Product Quantization-based KVCache for Long Context LLM Inferencecited by 93
  14. 2024Loki: Low-rank Keys for Efficient Sparse Attentioncited by 87
  15. 2019Explicit Sparse Transformer: Concentrated Attention Through Explicit Selectioncited by 61
  16. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Modelscited by 52
  17. 2024POD-Attention: Unlocking Full Prefill-Decode Overlap for Faster LLM Inferencecited by 46
  18. 2024Challenges in Deploying Long-Context Transformers: A Theoretical Peak Performance Analysiscited by 44
  19. 2025LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attentioncited by 41
  20. 2024TidalDecode: Fast and Accurate LLM Decoding with Position Persistent Sparse Attentioncited by 28
  21. 2021Energon: Towards Efficient Acceleration of Transformers Using Dynamic Sparse Attentioncited by 26
  22. 2023AccelTran: A Sparsity-Aware Accelerator for Dynamic Inference with Transformerscited by 24
  23. 2022A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipeliningcited by 21
  24. 2025SparseMM: Head Sparsity Emerges from Visual Concept Responses in MLLMscited by 21
  25. 2021Transformer Acceleration with Dynamic Sparse Attentioncited by 19
  26. 2025Long-Context Generalization with Sparse Attentioncited by 18
  27. 2024SOFA: A Compute-Memory Optimized Sparsity Accelerator via Cross-Stage Coordinated Tilingcited by 8
  28. 2024Vision Transformer with Sparse Scan Priorcited by 6
  29. 2025SPAD: Specialized Prefill and Decode Hardware for Disaggregated LLM Inferencecited by 6
  30. 2025DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Servingcited by 5
  31. 2025PADE: A Predictor-Free Sparse Attention Accelerator via Unified Execution and Stage Fusioncited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Elastic Threshold Attention: Learned Contextual Sparsity for Long-Context Decoding
  2. 2026AB-Sparse: Sparse Attention with Adaptive Block Size for Accurate and Efficient Long-Context Inference
  3. 2026Double-P: Hierarchical Top-P Sparse Attention for Long-Context LLMs
  4. 2026HieraSparse: Hierarchical Semi-Structured Sparse KV Attention
  5. 2026SwiftKV: An Edge-Oriented Attention Algorithm and Multi-Head Accelerator for Fast, Efficient LLM Decoding
  6. 2025Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving
  7. 2026Scaling Attention via Feature Sparsity
  8. 2026FAST-Prefill: FPGA Accelerated Sparse Attention for Long Context LLM Prefill ↗