mlsanityData to 7 Oct 2026

FibQuant: Universal Vector Quantization for Random-Access KV-Cache Compression

arXiv 2605.11478 ↗First posted 12 May 2026cs.AI cs.IT math.ITcited by 2

On the map: Efficient LLM inference and serving → KV cache quantization

Abstract (arXiv)

Long-context inference is increasingly a memory-traffic problem. The culprit is the key--value (KV) cache: it grows with context length, batch size, layers, and heads, and it is read at every decoding step. Rotation-based scalar codecs meet this systems constraint by storing a norm, applying a shared random rotation, and quantizing one coordinate at a time. They are universal and random-access, but they discard the geometry created by the normalization step. After a Haar rotation, a block of k consecutive coordinates is not a product source; it is a spherical-Beta source on the unit ball. We introduce textscFibQuant, a universal fixed-rate vector quantizer that keeps the same normalize--rotate--store interface while replacing scalar tables by a shared radial--angular codebook matched to this canonical source. The codebook combines Beta-quantile radii, Fibonacci,/,Roberts--Kronecker quasi-uniform directions, and multi-restart Lloyd--Max refinement. We prove that the resulting vector code strictly improves on its scalar product specialization at matched rate, with a high-rate gain that separates into a cell-shaping factor and a density-matching factor. The same construction gives a dense rate axis, including fractional-bit and sub-one-bit operating points, without calibration or variable-length addresses. On GPT-2 small KV caches, textscFibQuant traces a memory--fidelity frontier from 5× compression at 0.99 attention cosine similarity to 34× at 0.95. End-to-end on TinyLlama-1.1B, it is within 0.10 perplexity of fp16 at 4× compression and has 3.6× lower perplexity than scalar textscTurboQuant at b = 2 (8× compression), where scalar random-access quantization begins to fail.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 13 of its 28 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

13 of 28 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead5 Jun
  2. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization31 Jan

Other references · 11

  1. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  2. 2018Variational image compression with a scale hyperpriorcited by 910
  3. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  4. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  5. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  6. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 420
  7. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLMcited by 155
  8. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 116
  9. 2024RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Searchcited by 46
  10. 2025KVComp: A High-Performance, LLM-Aware, Lossy Compression Framework for KV Cachecited by 6
  11. 2026Revisiting RaBitQ and TurboQuant: A Symmetric Comparison of Methods, Theory, and Experimentscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Hurwitz Quaternion Multiplicative Quantization for KV Cache Compressioncited by 0
  2. 2026Fractal KV-Cache Archives: Lossless Symbolic Storage with In-Place Retrieval for Long-Context LLM Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026TurboAngle: Near-Lossless KV Cache Compression via Uniform Angle Quantization
  2. 2026Tailoring the Quantization Space for 1-Bit KV Cache Compression
  3. 2026OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond
  4. 2025CommVQ: Commutative Vector Quantization for KV Cache Compression
  5. 2025NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache
  6. 2025PolarQuant: Leveraging Polar Transformation for Efficient Key Cache Quantization and Decoding Acceleration
  7. 2026OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization
  8. 2026VQKV: High-Fidelity and High-Ratio Cache Compression via Vector-Quantization