mlsanityData to 14 Oct 2026

OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization

arXiv 2605.21226 ↗First posted 20 May 2026cs.LG cs.AIcited by 2

On the map: Efficient LLM inference and serving → KV cache quantization

Abstract (arXiv)

The key-value (KV) cache dominates memory bandwidth and footprint in long-context autoregressive inference. Recent rotation-preconditioned codecs (TurboQuant, PolarQuant) show that a structured random rotation followed by a per-coordinate scalar quantizer matched to an analytically tractable marginal is a near-optimal recipe for KV compression. OCTOPUS advances this paradigm through joint quantization of rotated coordinate triplets. Each triplet's direction is mapped to a square via an octahedral parameterization, and the two resulting coordinates and the triplet norm are Lloyd-Max quantized against implementation-matched marginals. Optimizing the per-triplet squared error gives a strictly non-uniform bit allocation depending only on the total dimensionality of the keys. We find the finite-dimensional quality optimum with sweeps to be constant on every real decoder we test. The codec is data-oblivious, online, and deterministic given a seed. Across text, video, and audio, OCTOPUS matches or beats every prior rotation codec at every reported bit width and metric, with a lead that grows as bits drop for extreme compression. Furthermore, a fused Triton implementation reconstructs keys on the fly without materializing the uncompressed key, so the codec adds no decode-time bandwidth or latency over the existing dequantization. Project Page: https://octopus-quant.github.io/

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 35 of its 49 references that are in the map. First posted on arXiv 20 May 2026.

Builds on

35 of 49 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025PolarQuant: Quantizing KV Caches with Polar Transformation4 Feb
  2. 2025PolarQuant: Leveraging Polar Transformation for Efficient Key Cache Quantization and Decoding Acceleration1 Feb
  3. 2025RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations25 Jan
  4. 2024QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead5 Jun
  5. 2024KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization7 May
  6. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs30 Mar
  7. 2024QAQ: Quality Adaptive Quantization for LLM KV Cache7 Mar
  8. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache5 Feb
  9. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization31 Jan
  10. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Serving29 Oct

Other references · 25

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  5. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  6. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  8. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  9. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  10. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  11. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  12. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
13 more
  1. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 519
  2. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 439
  3. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 433
  4. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
  5. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  6. 2024Data Engineering for Scaling Language Models to 128K Contextcited by 186
  7. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 184
  8. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLMcited by 158
  9. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 125
  10. 2024No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantizationcited by 101
  11. 2024WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains Morecited by 69
  12. 2024Lexico: Extreme KV Cache Compression via Sparse Coding over Universal Dictionariescited by 17
  13. 2024Efficient Autoregressive Audio Modeling via Next-Scale Predictioncited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Modelscited by 2
  2. 2026OmniKVQuant: KV Cache Quantization for Omni-LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026TurboAngle: Near-Lossless KV Cache Compression via Uniform Angle Quantization
  2. 2026ResidualQuant: KV Cache Quantization for Looped Transformers with 2-Bit Residuals
  3. 2026Sequential KV Cache Compression via Probabilistic Language Tries: Beyond the Per-Vector Shannon Limit
  4. 2026The risk of KV cache compression
  5. 2026Codec-Gauge: Learning Compression-Friendly Gauges for Transformer KV Caches
  6. 2026eOptShrinkQ: Near-Lossless KV Cache Compression Through Optimal Spectral Denoising and Quantization
  7. 2026IsoQuant: Hardware-Aligned SO(4) Isoclinic Rotations for LLM KV Cache Compression
  8. 2026FibQuant: Universal Vector Quantization for Random-Access KV-Cache Compression