mlsanityData to 7 Oct 2026

DASH-KV: Accelerating Long-Context LLM Inference via Asymmetric KV Cache Hashing

arXiv 2604.19351 ↗First posted 21 Apr 2026cs.CLcited by 1

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

The quadratic computational complexity of the standard attention mechanism constitutes a fundamental bottleneck for large language models in long-context inference. While existing KV cache compression methods alleviate memory pressure, they often sacrifice generation quality and fail to address the high overhead of floating-point arithmetic. This paper introduces DASH-KV, an innovative acceleration framework that reformulates attention as approximate nearest-neighbor search via asymmetric deep hashing. Under this paradigm, we design an asymmetric encoding architecture that differentially maps queries and keys to account for their distinctions in precision and reuse characteristics. To balance efficiency and accuracy, we further introduce a dynamic mixed-precision mechanism that adaptively retains full-precision computation for critical tokens. Extensive experiments on LongBench demonstrate that DASH-KV significantly outperforms state-of-the-art baseline methods while matching the performance of full attention, all while reducing inference complexity from O(N^2) to linear O(N).

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 36 of its 56 references that are in the map. First posted on arXiv 21 Apr 2026.

Builds on

36 of 56 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling4 Jun
  2. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  3. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs3 Oct
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep
  5. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time26 May

Other references · 31

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  4. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  5. 2020Reformer: The Efficient Transformercited by 1,658
  6. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  7. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  8. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  9. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  10. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  11. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  12. 2019Are Sixteen Heads Really Better than One?cited by 803
19 more
  1. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  2. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  3. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 255
  4. 2020Sparse Sinkhorn Attentioncited by 250
  5. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  6. 2021Scatterbrain: Unifying Sparse and Low-rank Attention Approximationcited by 128
  7. 2020Fast Transformers with Clustered Attentioncited by 123
  8. 2017HashNet: Deep Learning to Hash by Continuationcited by 121
  9. 2023The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reductioncited by 120
  10. 2016A Survey on Learning to Hashcited by 119
  11. 2018Approximate Nearest Neighbor Search in High Dimensionscited by 45
  12. 2015Simultaneous Feature Learning and Hash Coding with Deep Neural Networkscited by 44
  13. 2015Optimal Data-Dependent Hashing for Approximate Near Neighborscited by 43
  14. 2026Learning Global Hypothesis Space for Enhancing Synergistic Reasoning Chaincited by 15
  15. 2026Text summarization via global structure awarenesscited by 11
  16. 2026Language-Guided Token Compression with Reinforcement Learning in Large Vision-Language Modelscited by 9
  17. 2025Riemannian Optimization on Relaxed Indicator Matrix Manifoldcited by 5
  18. 2026GRASP: Guided Region-Aware Sparse Prompting for Adapting MLLMs to Remote Sensingcited by 5
  19. 2026Transforming External Knowledge into Triplets for Enhanced Retrieval in RAG of LLMscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026DimPO: Dimensionality Reduction for Attention using Preference Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ARKV: Adaptive and Resource-Efficient KV Cache Management under Limited Memory Budget for Long-Context Inference in LLMs
  2. 2025Spotlight Attention: Towards Efficient LLM Generation via Non-linear Hashing-based KV Cache Retrieval
  3. 2026ParisKV: Fast and Drift-Robust KV-Cache Retrieval for Long-Context LLMs
  4. 2025LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference
  5. 2026CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference
  6. 2024Compressing KV Cache for Long-Context LLM Inference with Inter-Layer Attention Similarity
  7. 2026C2KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference
  8. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference