# Dual-QK: Sharp Queries and Flat Keys for Prunable 2-bit KV Caches

arXiv: https://arxiv.org/abs/2610.09827 · HTML: https://mlsanity.com/paper/2610.09827/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving.
- Builds on 13 of its 47 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Placed in

- Efficient LLM inference and serving / [KV cache quantization](https://mlsanity.com/inference/kv-cache-quantization/)

## Builds on (most cited first, 13 of 13)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM](https://mlsanity.com/paper/2403.05527/) (2024)
- [Ministral 3](https://mlsanity.com/paper/2601.08584/) (2026)
- [Palu: Compressing KV-Cache with Low-Rank Projection](https://mlsanity.com/paper/2407.21118/) (2024)
- [Post-Training Sparse Attention with Double Sparsity](https://mlsanity.com/paper/2408.07092/) (2024)
- [SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models](https://mlsanity.com/paper/2405.06219/) (2024)
- [KVzap: Fast, Adaptive, and Faithful KV Cache Pruning](https://mlsanity.com/paper/2601.07891/) (2026)
- [OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization](https://mlsanity.com/paper/2605.17757/) (2026)
- [SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving](https://mlsanity.com/paper/2604.19157/) (2026)
- [Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms](https://mlsanity.com/paper/2608.04074/) (2026)

## Abstract (arXiv)

Long inputs and extended generation increase the storage and access costs of the key-value (KV) cache. Low-bit quantization reduces storage and memory traffic, while query-channel pruning can further reduce key-cache reads. Rotation-based quantization redistributes the energy of key outliers across channels. To maintain computational invariance, the same orthogonal transform must be applied to queries, preserving query-key dot products. However, this rotation can disperse query energy, weakening the separation between a few large components to retain and many small ones to prune. We introduce Dual-QK, which uses paired non-orthogonal query and key transforms to address this conflict. Using calibrated query and key statistics, Dual-QK combines partial key whitening with a query-aligned basis to balance key scales for INT2 quantization and concentrate query energy for dynamic channel pruning. Channel-0 protection and bucket-relative RoPE support low-bit accuracy over long contexts. Experiments on four models across five generative benchmarks and long-context retrieval tasks show improved accuracy over OSCAR on most tasks at 40% query-channel sparsity. At a 128K context, Dual-QK provides 6.8× KV-cache compression and an estimated 8.3× reduction in KV read volume relative to unpruned BF16. Under the evaluated configurations, our SGLang implementation achieves up to 3.75× the decoding throughput of unpruned BF16.