mlsanityData to 14 Oct 2026

SparPR: Runtime KV-Cache Recovery for Resource-Constrained LLM Inference on Consumer Hardware

arXiv 2610.18140 ↗First posted 13 Oct 2026cs.CLcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization also fits: Serving systems and request scheduling

Abstract (arXiv)

Running large language models (LLMs) on consumer hardware is bottlenecked by the key-value (KV) cache, which grows with every decoded token and quickly exhausts the memory and bandwidth of a single consumer GPU. We present SparPR (Sparse Partial Recomputation), a runtime KV-cache recovery mechanism that does not modify model weights and requires no offline training. SparPR couples three online decisions: Query-Aware key-block selection scores historical blocks against the current query; dynamic budget control estimates per-block copy and recompute costs to decide how many blocks to retain; and on-demand local recomputation with cost-aware fallback merges retained blocks into continuous spans and switches between sparse recomputation and full recovery according to a runtime cost comparison. Implemented on FlexGen/FlexLLMGen, SparPR is evaluated on a single consumer laptop (RTX 3070 Ti 8GB, Ryzen 9 6900HX, 16GB RAM). It cuts end-to-end latency by 31%, 52%, and 63% at 1k, 2k, and 4k prompt tokens respectively, with 1.4x-2.7x higher throughput, and completes an 8k-token workload in 44 minutes where the FlexGen baseline times out. Under a fixed cache budget, SparPR retains a Needle Accuracy that stays well above StreamingLLM and H2O at every keep ratio, and ablation studies confirm that the three mechanisms act jointly.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 9 of its 11 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

9 of 11 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026ProphetKV: User-Query-Driven Selective Recomputation for Efficient KV Cache Reuse in Retrieval-Augmented Generation31 Jan
  2. 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference28 Oct
  3. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference16 Jun
  4. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 5

  1. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  4. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  5. 2025Breaking the Boundaries of Long-Context LLM Inference: Adaptive KV Management on a Single Commodity GPUcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026KV Cache Optimization Strategies for Scalable and Efficient LLM Inference
  2. 2025LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference
  3. 2024KVPR: Efficient LLM Inference with I/O-Aware KV Cache Partial Recomputation
  4. 2025BaKlaVa -- Budgeted Allocation of KV cache for Long-context Inference
  5. 2025EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Serving
  6. 2025FreeKV: Boosting KV Cache Retrieval for Efficient LLM Inference
  7. 2025LLaMCAT: Optimizing Large Language Model Inference with Cache Arbitration and Throttling
  8. 2025Efficient LLM Inference with Activation Checkpointing and Hybrid Caching