mlsanityData to 14 Oct 2026

Can I Buy Your KV Cache?

arXiv 2606.13361 ↗First posted 11 Jun 2026cs.AI cs.CE cs.MAcited by 0

Not on the map. Why?

Abstract (arXiv)

Right now, across the world, AI agents are repeating the same absurd act: to read one document, they each recompute it from scratch. Every agent re-runs prefill, the most compute-intensive step a large model takes, over identical text, only to rebuild a key-value (KV) cache identical to the one the agent before it just built. The same answer, computed a million times. We make a proposal that is almost offensively simple: compute it once. Let a publisher precompute a document's KV cache, and let every other agent buy the right to load it and skip prefill. It works, and it is token-exact: loading a precomputed KV and continuing matches prefilling from scratch (24/24 greedy tokens, and at the logits level), with no accuracy cost. On Qwen3-4B, reuse is 9-50x cheaper in compute than prefill, and the gap widens with length (prefill's attention scales with L2), so a single reuse already pays it back. Then the part that matters: where the KV lives. Shipping it fails, because KV is nearly incompressible, so per-load egress costs more than the prefill it saves. Hosting it provider-side, exactly as production prompt-caching works, removes egress entirely. The size of the prize is set by our measured compute saving: serving one hot 3774-token document to 80M agents costs ~1.5M to re-prefill but only ~0.03M of reuse compute (49.7x less). The 0.1x cache-read tariff APIs charge passes a 10x discount to users while sitting inside this measured envelope, so the 10x is a floor that the measured ~50x compute saving clears, and the gap to the physical ~50x is provider margin: millions of dollars per popular document. We frame the resulting agent-native prefill CDN and leave lossless KV compression and a cross-party payment layer as the open problems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 9 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

9 of 9 references.

References · 9

  1. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 223
  2. 2025Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generationcited by 36
  3. 2024InstInfer: In-Storage Attention Offloading for Cost-Effective Long-Context LLM Inferencecited by 19
  4. 2025CacheClip: Accelerating RAG with Effective KV Cache Reusecited by 16
  5. 2025MatKV: Trading Compute for Flash Storage in LLM Inferencecited by 2
  6. 2026Observation, Not Prediction: Conversation-Level Disaggregated Scheduling for Agentic Servingcited by 2
  7. 2026NetKV: Network-Aware Decode Instance Selection for Disaggregated LLM Inferencecited by 1
  8. 2026SpectrumKV: Per-Token Mixed-Precision KV Cache Transfer for Prefill-Decode Disaggregated LLM Servingcited by 1
  9. 2026SIFT: Selective-Index For Fast Compute of RAG Prefill by Exploiting Attention Invariancecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Models Take Notes at Prefill: KV Cache Can Be Editable and Composable
  2. 2026KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints
  3. 2026Request Order Matters: Cache-History Sensitivity in Selective KV-Cache Reuse for Rolling Agents
  4. 2026What It Costs to Compose, Rebuild, and Correct Precomputed Memory
  5. 2026Elastic KV Cache for LLM Serving:A Working Reclamation Mechanism, and Why Chunked Prefill Already Closes the Gap
  6. 2026EfficientAgent: What Makes KV Cache Offloading Work for Concurrent Agents?
  7. 2026KV Packet: Recomputation-Free Context-Independent KV Caching for LLMs
  8. 2026PAGE: Partition-Aware Gated KV-Cache Eviction