mlsanityData to 14 Oct 2026

Squeezing the Cache, Preserving the Truth: Monotonic Equipotential Allocation with Geodesia-KV

arXiv 2608.23599 ↗First posted 19 Aug 2026cs.AR cs.CLcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Current assessment of KV-cache compression performance confuses resident bits with read bandwidth and is affected by the artifacts of chunked teacher-forcing. We present Geodesia-KV, a family of training-free KV cache policies based on monotonic block-wise precision allocation, exact rate-distortion residuals, and query-sparse reading, enabling proper hardware-ready compression. With proper separation of resident and read bits and causal evaluation, we show that Geodesia-KV significantly outperforms other approaches. Specifically, on WikiText-2 with 16k context, the 5-bit operating point of Geodesia-KV results in lower perplexity at lower bitrate than KIVI-4 on Qwen. In addition, our compressed-Quest version delivers improved perplexity and reduces resident (9.83 vs 16.25 bits/value) and read rates (1.95 vs 2.32 bits/value) over baseline sparse methods on PG-19. As Geodesia-KV is implemented as native GeodesiaKVCacheManager plug-in of vLLM, Geodesia-KV fully removes the need for dense cache residency via monotonic bit demotion. With the full consumer hardware evaluation, Geodesia-KV leads to 1M-token context generation on a single 16 GiB GPU with up to 71.7% peak VRAM savings on all leading architectures (Qwen, Llama, DeepSeek).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 29 of its 35 references that are in the map. First posted on arXiv 19 Aug 2026.

Builds on

29 of 35 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferences16 Mar
  2. 2025InfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPU13 Feb
  3. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference16 Jul
  4. 2024D2O: Dynamic Discriminative Operations for Efficient Long-Context Inference of Large Language Models18 Jun
  5. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference16 Jun
  6. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling4 Jun
  7. 2024MiniCache: KV Cache Compression in Depth Dimension for Large Language Models23 May
  8. 2024PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference21 May
  9. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  10. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM8 Mar

Other references · 19

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2024Qwen2.5 Technical Reportcited by 4,942
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  4. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  5. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  7. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  8. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  9. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  10. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 726
  11. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  12. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
7 more
  1. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  2. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
  3. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 383
  4. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  5. 2024Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attentioncited by 200
  6. 2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memorycited by 197
  7. 2024RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrievalcited by 116

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation
  2. 2026KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMs
  3. 2026Every Cache Entry Earns Its Place: Global Allocation of Resolution and Coverage for KV Cache Compression
  4. 2026CORE: COverage CAlibration and Evicted-Mass REdistribution for KV Cache
  5. 2026The risk of KV cache compression
  6. 2026TurboAngle: Near-Lossless KV Cache Compression via Uniform Angle Quantization
  7. 2025Compactor: Calibrated Query-Agnostic KV Cache Compression with Approximate Leverage Scores
  8. 2025LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation