mlsanityData to 7 Oct 2026

Codec-Gauge: Learning Compression-Friendly Gauges for Transformer KV Caches

arXiv 2607.20538 ↗First posted 10 Jul 2026cs.LG cs.AIcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Long-context Transformer inference increasingly relies on KV-cache compression or quantization. Prior rotation and transform-coding results suggest that the channel basis of each key/value vector affects how faithfully a fixed backend preserves model behavior. We introduce Codec-Gauge, a post-training cache-coordinate layer that learns small orthogonal channel transforms around existing compression and quantization backends. Its frequency-distribution objective combines a token-channel DCT spectral-centroid loss with a smooth rate proxy to concentrate KV energy in low-frequency codec-facing layouts. We evaluate actual compression and decompression using measured bytes and rolling compressed-history scoring. Across six models at 3, 4, and 6 bits/value, learned gauges reduce zfp KL divergence by 44.0% on average relative to raw coordinates and outperform random, Hadamard, DCT, and PCA/KLT controls. The same gauges improve quality preservation for block-uniform and KIVI-style quantization. Experiments on a 27B model and long-context task prompts reproduce the quality trend, while serial storage and timing measurements validate the implemented compressed-cache paths. These results establish cache-coordinate geometry as a practical post-training variable for improving compression fidelity without changing model weights, attention semantics, or backend coding rules.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 26 of its 32 references that are in the map. First posted on arXiv 10 Jul 2026.

Builds on

26 of 32 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025FAEDKV: Infinite-Window Fourier Transform for Unbiased KV Cache Compression26 Jul
  2. 2025FreqKV: Key-Value Compression in Frequency Domain for Context Window Extension1 May
  3. 2025RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression19 Feb
  4. 2024MatryoshkaKV: Adaptive KV Compression via Trainable Orthogonal Projection16 Oct
  5. 2024Palu: Compressing KV-Cache with Low-Rank Projection30 Jul
  6. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management28 Jun
  7. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling4 Jun
  8. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  9. 2024No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization28 Feb
  10. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving11 Oct
  11. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs3 Oct
  12. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep

Other references · 14

  1. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  2. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  3. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  4. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  5. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  6. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  7. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  8. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  9. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 367
  10. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 200
  11. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 116
  12. 2024Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inferencecited by 115
2 more
  1. 2025RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotationscited by 42
  2. 2025KVLinC : KV Cache Quantization with Hadamard Rotation and Linear Correctioncited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Sequential KV Cache Compression via Probabilistic Language Tries: Beyond the Per-Vector Shannon Limit
  2. 2026KV Cache Compression Through the Lens of Transform Coding
  3. 2026The risk of KV cache compression
  4. 2026OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization
  5. 2026eOptShrinkQ: Near-Lossless KV Cache Compression Through Optimal Spectral Denoising and Quantization
  6. 2026How Much Cache Does Reasoning Need? Depth-Cache Tradeoffs in KV-Compressed Transformers
  7. 2026The Residual Stream Is All You Need: On the Redundancy of the KV Cache in Transformer Inference
  8. 2025Reconstructing KV Caches with Cross-layer Fusion For Enhanced Transformers