mlsanityData to 7 Oct 2026

CHESS: Context-aware Hierarchical Efficient Semantic Selection for Long-Context LLM Inference

arXiv 2602.20732 ↗First posted 24 Feb 2026cs.AIcited by 1

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Long-context LLMs demand accurate inference at low latency, yet decoding becomes primarily constrained by KV cache as context grows. Prior pruning methods are largely context-agnostic: their token selection ignores step-wise relevance and local semantics, which undermines quality. Moreover, their irregular accesses and selection overheads yield only limited wall-clock speedups. To address this, we propose CHESS, an textitalgorithm-system co-design KV-cache management system. Algorithmically, CHESS introduces a context-aware, hierarchical selection policy that dynamically reconstructs a coherent context for the current decoding. System-wise, coarse granularity selection eliminates expensive data movement, fully realizing practical acceleration from theoretical sparsity. Extensive evaluations demonstrate that CHESS surpasses Full-KV quality using only 1% of the KV cache, delivers low-latency stable inference with up to 4.56× higher throughput, and consistently outperforms other strong baselines. Code is available at hrefhttps://anonymous.4open.science/r/CHESS-9958/https://anonymous.4open.science/r/CHESS/.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 22 of its 23 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

22 of 23 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025EvolKV: Evolutionary KV Cache Compression for LLM Inference10 Sep
  2. 2025KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments21 Apr
  3. 2025RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression19 Feb
  4. 2025ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference1 Feb
  5. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving2 Jan
  6. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference16 Jun
  7. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  8. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs3 Oct
  9. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep
  10. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 12

  1. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  2. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  4. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  5. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  6. 2024AFlow: Automating Agentic Workflow Generationcited by 395
  7. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 367
  8. 2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memorycited by 192
  9. 2022Can Foundation Models Wrangle Your Data?cited by 173
  10. 2025A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applicationscited by 40
  11. 2025NexusSum: Hierarchical LLM Agents for Long-Form Narrative Summarizationcited by 10
  12. 2023Dialogizer: Context-aware Conversational-QA Dataset Generation from Textual Sourcescited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026KVDiagnosis: A Diagnostic Benchmark for KV-Cache Compression in Long-Context Language Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026ARKV: Adaptive and Resource-Efficient KV Cache Management under Limited Memory Budget for Long-Context Inference in LLMs
  2. 2024TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection
  3. 2025QuickSilver -- Speeding up LLM Inference through Dynamic Token Halting, KV Skipping, Contextual Token Fusion, and Adaptive Matryoshka Quantization
  4. 2025WindowKV: Task-Adaptive Group-Wise KV Cache Window Selection for Efficient LLM Inference
  5. 2026ParisKV: Fast and Drift-Robust KV-Cache Retrieval for Long-Context LLMs
  6. 2025LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference
  7. 2026DualDecoder: Accelerate Long Context LLM Inference by Predictive Prefetch
  8. 2024SCBench: A KV Cache-Centric Analysis of Long-Context Methods