mlsanityData to 7 Oct 2026

Rotate, Clip, and Partition: Towards W2A4KV4 Quantization by Integrating Rotation and Learnable Non-uniform Quantizer

arXiv 2502.15779 ↗First posted 17 Feb 2025cs.LG cs.AI cs.CLcited by 3

On the map: Efficient LLM inference and serving → KV cache quantization

Abstract (arXiv)

We propose Rotate, Clip, and Partition (RCP), a quantization-aware training (QAT) approach that first realizes extreme compression of LLMs with W2A4KV4(2-bit weight, 4-bit activation, and 4-bit KV cache) configuration. RCP integrates recent rotation techniques with a novel non-uniform weight quantizer design, by quantitatively analyzing the impact of random rotation on 2-bit weight quantization. Our weight quantizer features Learnable Direct Partitioning (LDP), which introduces learnable parameters to directly learn non-uniform intervals jointly with LLM weights. We also present a specialized GPU kernel that supports GEMV on non-uniform W2A4. Experiments show that RCP can compress LLaMA-2-7B to W2A4KV4 with a loss of only 2.84 WikiText2 ppl and 5.29 times reduced memory footprint. Furthermore, RCP can quantize challenging mobile-targeted LLaMA-3.2 models and domain-specific WizardCoder-7B and MetaMath-7B with no critical problems such as convergence failure and repetition. Code is available at https://github.com/ songsm921/RCP.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 25 of its 38 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

25 of 38 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024SpinQuant: LLM quantization with learned rotations26 May
  2. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs30 Mar

Other references · 23

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  9. 2016Pointer Sentinel Mixture Modelscited by 2,992
  10. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  11. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  12. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
11 more
  1. 2019Learned Step Size Quantizationcited by 604
  2. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 507
  3. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  4. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 378
  5. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
  6. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 165
  7. 2024BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillationcited by 80
  8. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 78
  9. 2024Fast Matrix Multiplications for Lookup Table-Quantized LLMscited by 27
  10. 2024AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inferencecited by 17
  11. 2023AFPQ: Asymmetric Floating Point Quantization for LLMscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMscited by 3
  2. 2026Qift: Shift-Friendly No-Zero W2 Post-Training Quantization for Rotated W2A4/KV4 LLM Inferencecited by 0
  3. 2026OffQ: Taming Structured Outliers in LLM Quantization by Offsettingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Few Bits, One Law: Toward W2A4KV2
  2. 2026OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
  3. 2025RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations
  4. 2026Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs
  5. 2026SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving
  6. 2026OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond
  7. 2026FibQuant: Universal Vector Quantization for Random-Access KV-Cache Compression
  8. 2026RAMP: Reinforcement Adaptive Mixed Precision Quantization for Efficient On Device LLM Inference