mlsanityData to 14 Oct 2026

SPHERICAL KV: Angle-Domain Attention and Rate-Distortion Retention for Efficient Long-Context Inference

arXiv 2605.18856 ↗First posted 13 May 2026cs.LG cs.CL cs.ITcited by 2

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Long-context inference is increasingly constrained by the KV cache: resident memory grows with context length, and decoding becomes limited by repeated High Bandwidth Memory (HBM) streaming rather than arithmetic. Existing methods such as eviction, windowing, quantization, and offloading reduce footprint, but often leave the critical-path bottleneck only partially addressed, especially when compressed states must still be reconstructed into dense vectors during decoding. We present Spherical KV, a long-context inference method that treats KV allocation as a rate-distortion problem grounded in attention geometry for efficient decoding. The method is built on two ideas: (i) represent directional information cheaply in the decode hot loop, and (ii) allocate retention and precision according to estimated future utility. Its first component, Angle-Domain Attention (ADA), stores keys in a spherical parameterization consisting of a scalar radius and compact angle codes, and computes attention logits directly from these codes without reconstructing dense keys. This preserves a paged, block-local, fusion-friendly decode path and directly targets HBM traffic in realistic serving settings. Its second component, Rate-Distortion Retention (RDR), jointly chooses keep/drop decisions and precision tiers per token and head under a fixed budget, producing tier-homogeneous pages with lightweight metadata and coalesced reads. Together, ADA and RDR provide a deployment-oriented mechanism for reducing KV residency while preserving decode efficiency.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 35 of its 49 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

35 of 49 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs3 Dec
  2. 2025FlexiCache: Leveraging Temporal Stability of Attention Heads for Efficient KV Cache Management2 Nov
  3. 2025LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference8 Oct
  4. 2025Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution1 Oct
  5. 2025GraphKV: Breaking the Static Selection Paradigm with Graph-Based KV Cache Eviction30 Aug
  6. 2025BitDecoding: Unlocking Tensor Cores for Long-Context LLMs with Low-Bit KV Cache24 Mar
  7. 2025LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference11 Mar
  8. 2025ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference1 Feb
  9. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving2 Jan
  10. 2024SCBench: A KV Cache-Centric Analysis of Long-Context Methods13 Dec
  11. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling4 Jun
  12. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
4 more
  1. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM8 Mar
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep
  3. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun
  4. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time26 May

Other references · 19

  1. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  3. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  5. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  7. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  8. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  9. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 726
  10. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  11. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  12. 2024ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identificationcited by 93
7 more
  1. 2025PolarQuant: Quantizing KV Caches with Polar Transformationcited by 27
  2. 2025Accurate KV Cache Quantization with Outlier Tokens Tracingcited by 18
  3. 2025PolarQuant: Leveraging Polar Transformation for Efficient Key Cache Quantization and Decoding Accelerationcited by 10
  4. 2025Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boostcited by 10
  5. 2025Value-Guided KV Compression for LLMs via Approximated CUR Decompositioncited by 7
  6. 2025Hold Onto That Thought: Assessing KV Cache Compression On Reasoningcited by 6
  7. 2025KV Admission: Learning What to Write for Efficient Long-Context LLM Inferencecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Hurwitz Quaternion Multiplicative Quantization for KV Cache Compressioncited by 0
  2. 2026Spend Bytes on Breadth: Precision-Count Trade-offs for Decode-Time KV Compression in Long Chain-of-Thought Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026KVBuffer: IO-aware Serving for Linear Attention
  2. 2026SPLASH: Co-Designing Sparse Attention with High-Bandwidth Flash for Efficient Long-Context Inference
  3. 2026Lynx: Progressive Speculative Quantization for accelerating KV Transfer in Long-Context Inference
  4. 2026OasisKV: Scaling In-Decode KV Cache Beyond HBM with Lookahead Sparse Prefetching
  5. 2026Distance-KV: Exploiting Relative Distance for Efficient Long-Context Inference
  6. 2026ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inference
  7. 2026SAC: Disaggregated KV Cache System for Sparse Attention LLMs with CXL
  8. 2026Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving