mlsanityData to 14 Oct 2026

PN-Mix: Pair-Norm Guided Mixed-Precision Key Cache Quantization for Efficient Long-Output LLM Inference

arXiv 2610.19605 ↗First posted 14 Oct 2026cs.LG cs.CLcited by 0

On the map: Efficient LLM inference and serving → KV cache quantization

Abstract (arXiv)

Long-output inference in large language models suffers from severe KV cache memory and bandwidth bottlenecks. While KV cache quantization alleviates this issue, low-bit quantization (e.g., 2-bit) severely degrades model accuracy. Recent mixed-precision key-cache quantization methods mitigate this degradation by allocating higher precision to important key channels, but estimate channel saliency using only key statistics. Through attention logit perturbation analysis at the RoPE channel-pair level, we show that quantization errors in channel pairs with large query--key pair-norm products contribute significantly to logit variation. Based on this insight, we propose PN-Mix, a mixed-precision key-cache quantization method for RoPE-based decoder-only LLMs, which allocates higher precision to salient channel pairs identified once during the prefill stage and reused throughout decoding. Across standard reasoning, code-generation, and AIME benchmarks, PN-Mix with 2.26-bit average precision achieves comparable or better average accuracy than prior mixed-precision methods using 2.32--2.39 bits. Furthermore, in our Pareto sweep, PN-Mix degrades gracefully down to 1.5 bits, whereas the compared baselines exhibit sharper performance drops below 2.25 bits.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 28 of its 38 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

28 of 38 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boost23 Nov
  2. 2025PM-KVQ: Progressive Mixed-precision KV Cache Quantization for Long-CoT LLMs24 May
  3. 2025Accurate KV Cache Quantization with Outlier Tokens Tracing16 May
  4. 2025Quantize What Counts: More for Keys, Less for Values20 Feb
  5. 2025KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference6 Feb
  6. 2025RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations25 Jan
  7. 2024QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead5 Jun
  8. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache5 Feb
  9. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization31 Jan

Other references · 19

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  6. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  8. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  9. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  10. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  11. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  12. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 367
7 more
  1. 2024No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantizationcited by 101
  2. 2024Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Taskscited by 96
  3. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Modelscited by 73
  4. 2024ThinK: Thinner Key Cache by Query-Driven Pruningcited by 68
  5. 2024Palu: Compressing KV-Cache with Low-Rank Projectioncited by 64
  6. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Modelscited by 54
  7. 2025LeanK: Learnable K Cache Channel Pruning for Efficient Decodingcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SemKV: Semantic Mixed-Precision KV Cache Quantization Guided by the Quality Cliff for Long-Context LLM Inference
  2. 2025Cocktail: Chunk-Adaptive Mixed-Precision Quantization for Long-Context LLM Inference
  3. 2024Progressive Mixed-Precision Decoding for Efficient LLM Inference
  4. 2026Don't Waste Bits! Adaptive KV-Cache Quantization for Lightweight On-Device LLMs
  5. 2025KVmix: Gradient-Based Layer Importance-Aware Mixed-Precision Quantization for KV Cache
  6. 2025MoQAE: Mixed-Precision Quantization for Long-Context LLM Inference via Mixture of Quantization-Aware Experts
  7. 2026OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond
  8. 2026Tailoring the Quantization Space for 1-Bit KV Cache Compression