mlsanityData to 14 Oct 2026

ParisKV: Fast and Drift-Robust KV-Cache Retrieval for Long-Context LLMs

arXiv 2602.07721 ↗First posted 7 Feb 2026cs.LG cs.CL cs.DBcited by 2

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

KV-cache retrieval is essential for long-context LLM inference, yet existing methods struggle with distribution drift and high latency at scale. We introduce ParisKV, a drift-robust, GPU-native KV-cache retrieval framework based on collision-based candidate selection, followed by a quantized inner-product reranking estimator. For million-token contexts, ParisKV supports CPU-offloaded KV caches via Unified Virtual Addressing (UVA), enabling on-demand top-k fetching with minimal overhead. ParisKV matches or outperforms full attention quality on long-input and long-generation benchmarks. It achieves state-of-the-art long-context decoding efficiency: it matches or exceeds full attention speed even at batch size 1 for long contexts, delivers up to 2.8× higher throughput within full attention's runnable range, and scales to million-token contexts where full attention runs out of memory. At million-token scale, ParisKV reduces decode latency by 17× and 44× compared to MagicPIG and PQCache, respectively, two state-of-the-art KV-cache Top-k retrieval baselines, code is available at https://github.com/amy-77/ParisKV/tree/main.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 51 of its 64 references that are in the map. First posted on arXiv 7 Feb 2026.

Builds on

51 of 64 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution1 Oct
  2. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction29 May
  3. 2025FreeKV: Boosting KV Cache Retrieval for Efficient LLM Inference19 May
  4. 2025KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments21 Apr
  5. 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference28 Oct
  6. 2024Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoning25 Oct
  7. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads14 Oct
  8. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference16 Jul
  9. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference16 Jun
  10. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling4 Jun
  11. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  12. 2024Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inference14 Mar
1 more
  1. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 38

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  6. 2023Mistral 7Bcited by 3,164
  7. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  8. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  9. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  10. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  11. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  12. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
26 more
  1. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  2. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  3. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  4. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
  5. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 383
  6. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
  7. 2025MoBA: Mixture of Block Attention for Long-Context LLMscited by 198
  8. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  9. 2025XAttention: Block Sparse Attention with Antidiagonal Scoringcited by 137
  10. 2024SeerAttention: Learning Intrinsic Sparse Attention in Your LLMscited by 124
  11. 2025FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inferencecited by 123
  12. 2024RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrievalcited by 116
  13. 2024MagicPIG: LSH Sampling for Efficient LLM Generationcited by 111
  14. 2024PQCache: Product Quantization-based KVCache for Long Context LLM Inferencecited by 95
  15. 2025Twilight: Adaptive Attention Sparsity with Hierarchical Top-p Pruningcited by 49
  16. 2025KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inferencecited by 40
  17. 2023Similarity search in the blink of an eye with compressed indicescited by 21
  18. 2025VisJudge-Bench: Aesthetics and Quality Assessment of Visualizationscited by 16
  19. 2025EraRAG: Efficient and Incremental Retrieval Augmented Generation for Growing Corporacited by 15
  20. 2025LEAD: Iterative Data Selection for Efficient LLM Instruction Tuningcited by 13
  21. 2025CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Datacited by 13
  22. 2024DET-LSH: A Locality-Sensitive Hashing Scheme with Dynamic Encoding Tree for Approximate Nearest Neighbor Searchcited by 6
  23. 2021Fast Data Series Indexing for In-Memory Datacited by 4
  24. 2025SAQ: Pushing the Limits of Vector Quantization through Code Adjustment and Dimension Segmentationcited by 3
  25. 2026SOCKET: SOft Collision Kernel EsTimator for Sparse Attentioncited by 3
  26. 2024Fast and Exact Similarity Search in less than a Blink of an Eyecited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026PQ-HSA: Reusing Product-Quantized Scores for Hybrid Sparse-Approximate Attentioncited by 0
  2. 2026Accelerating Cross-Model LLM Inference via Attention Map Reusecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025LouisKV: Efficient KV Cache Retrieval for Long Input-Output Sequences
  2. 2026SeKV: Resolution-Adaptive KV Cache with Hierarchical Semantic Memory for Long-Context LLM Inference
  3. 2026KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference
  4. 2025RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression
  5. 2025LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference
  6. 2026CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference
  7. 2026DualDecoder: Accelerate Long Context LLM Inference by Predictive Prefetch
  8. 2026ARKV: Adaptive and Resource-Efficient KV Cache Management under Limited Memory Budget for Long-Context Inference in LLMs