RAP: KV-Cache Compression via RoPE-Aligned Pruning
On the map: Efficient LLM inference and serving → KV cache optimization
Abstract (arXiv)
Long-context inference in large language models (LLMs) is bottlenecked by the memory and compute of the key-value (KV) cache. Structured pruning is a direct way to shrink it: dropping the least useful channels of the Wk, Wv projection weights to reduce the output KV dimensions. However, modern LLMs apply Rotary Position Embedding (RoPE) after the QK projections, which rotates feature dimensions in pairs. Therefore, removing individual channels breaks these pairs, corrupting RoPE's positional semantics and rendering the pruned model unusable. We propose RoPE-Aligned Pruning (RAP), which constrains the pruning granularity to RoPE-aligned pairs rather than individual channels: removing whole pairs to keep the rotation intact. Our evaluation across Llama, Mistral, and Qwen models from 3B to 14B shows that RAP preserves accuracy at 30% KV compression (retain ratio rho = 0.7), far outperforms RoPE-blind channel pruning, stays near the strongest low-rank method at lower attention cost, and composes with orthogonal methods such as quantization.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 43 of its 56 references that are in the map. First posted on arXiv 1 Feb 2026.
Builds on
43 of 56 references.
Closest ancestors · 10
Same direction, newest first
- 2025SparK: Query-Aware Unstructured Sparsity with Recoverable KV Cache Channel Pruning21 Aug
- 2025ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibration30 May
- 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction29 May
- 2025EliteKV: Scalable KV Cache Compression via RoPE Frequency Selection and Joint Low-Rank Projection3 Mar
- 2024SCBench: A KV Cache-Centric Analysis of Long-Context Methods13 Dec
- 2024MatryoshkaKV: Adaptive KV Compression via Trainable Orthogonal Projection16 Oct
- 2024Palu: Compressing KV-Cache with Low-Rank Projection30 Jul
- 2024ThinK: Thinner Key Cache by Query-Driven Pruning30 Jul
- 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
- 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun
Other references · 33
- 2017Attention Is All You Needcited by 59,932
- 2023GPT-4 Technical Reportcited by 18,132
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
- 2020Longformer: The Long-Document Transformercited by 3,362
- 2023Qwen Technical Reportcited by 3,273
- 2016Pointer Sentinel Mixture Modelscited by 2,992
- 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
21 more
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
- 2020Big Bird: Transformers for Longer Sequencescited by 1,673
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
- 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
- 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
- 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
- 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
- 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
- 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
- 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
- 2019Importance Estimation for Neural Network Pruningcited by 531
- 2024SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compressioncited by 231
- 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 200
- 2022A Fast Post-Training Pruning Framework for Transformerscited by 161
- 2024AI and Memory Wallcited by 89
- 2024Challenges in Deploying Long-Context Transformers: A Theoretical Peak Performance Analysiscited by 44
- 2025Circle-RoPE: Cone-like Decoupled Rotary Positional Embedding for Large Vision-Language Modelscited by 14
- 2025Does quantization affect models' performance on long-context tasks?cited by 10
- 2025LeanK: Learnable K Cache Channel Pruning for Efficient Decodingcited by 5
- 2025Lethe: Layer- and Time-Adaptive KV Cache Pruning for Reasoning-Intensive LLM Servingcited by 5
- 2025The Rotary Position Embedding May Cause Dimension Inefficiency in Attention Heads for Long-Distance Retrievalcited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026DepthKV: Layer-Dependent KV Cache Pruning for Long-Context LLM Inference
- 2025KVCompose: Efficient Structured KV Cache Compression with Composite Tokens
- 2024More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression
- 2025RAP: Runtime Adaptive Pruning for LLM Inference
- 2026A Self-Pruning Transformer: Extreme KV-Cache Compression with Universal Attention
- 2026PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression
- 2026VisCache: Visual KV Cache Pruning for Efficient Vision Large Language Model Inference
- 2025Pruning Laws for Large Language Models