# MEPIC: Memory Efficient Position Independent Caching for LLM Serving

arXiv: https://arxiv.org/abs/2512.16822 · HTML: https://mlsanity.com/paper/2512.16822/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation.
- Builds on 27 of its 41 references that are in the map.
- First posted on arXiv 18 Dec 2025.

## Placed in

- Retrieval-augmented generation / [Generation and context integration](https://mlsanity.com/rag/generation-and-context-integration/)
- Efficient LLM inference and serving / [KV cache optimization](https://mlsanity.com/inference/kv-cache-optimization/)

## Builds on (most cited first, 27 of 27)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [SGLang: Efficient Execution of Structured Language Model Programs](https://mlsanity.com/paper/2312.07104/) (2023)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization](https://mlsanity.com/paper/2401.18079/) (2024)
- [Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs](https://mlsanity.com/paper/2310.01801/) (2023)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [Prompt Cache: Modular Attention Reuse for Low-Latency Inference](https://mlsanity.com/paper/2311.04934/) (2023)
- [CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion](https://mlsanity.com/paper/2405.16444/) (2024)
- [CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving](https://mlsanity.com/paper/2310.07240/) (2023)
- [Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention](https://mlsanity.com/paper/2403.19708/) (2024)
- [emrQA: A Large Corpus for Question Answering on Electronic Medical Records](https://mlsanity.com/paper/1809.00732/) (2018)
- [LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference](https://mlsanity.com/paper/2510.09665/) (2025)
- [RazorAttention: Efficient KV Cache Compression Through Retrieval Heads](https://mlsanity.com/paper/2407.15891/) (2024)
- [ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification](https://mlsanity.com/paper/2405.14256/) (2024)
- [QAQ: Quality Adaptive Quantization for LLM KV Cache](https://mlsanity.com/paper/2403.04643/) (2024)
- [KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse](https://mlsanity.com/paper/2502.16002/) (2025)
- [Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live](https://mlsanity.com/paper/2511.02230/) (2025)
- [KVCache Cache in the Wild: Characterizing and Optimizing KVCache Cache at a Large Cloud Provider](https://mlsanity.com/paper/2506.02634/) (2025)
- [KVShare: An LLM Service System with Efficient and Effective Multi-Tenant KV Cache Reuse](https://mlsanity.com/paper/2503.16525/) (2025)
- [CacheClip: Accelerating RAG with Effective KV Cache Reuse](https://mlsanity.com/paper/2510.10129/) (2025)
- [ContextPilot: Fast Long-Context Inference via Context Reuse](https://mlsanity.com/paper/2511.03475/) (2025)
- [A3: Attention-Aware Accurate KV Cache Fusion for Fast Large Language Model Serving](https://mlsanity.com/paper/2511.17560/) (2025)
- [CoDec: Prefix-Shared Decoding Kernel for LLMs](https://mlsanity.com/paper/2505.17694/) (2025)
- [Generative Caching for Structurally Similar Prompts and Responses](https://mlsanity.com/paper/2511.17565/) (2025)
- [MoSKA: Mixture of Shared KV Attention for Efficient Long-Sequence LLM Inference](https://mlsanity.com/paper/2511.06010/) (2025)

## Built on by (most cited first, 11 of 11)

- [Irminsul: MLA-Native Position-Independent Caching for Agentic LLM Serving](https://mlsanity.com/paper/2605.05696/) (2026)
- [LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs](https://mlsanity.com/paper/2608.11231/) (2026)
- [LazyAttention: Efficient Retrieval-Augmented Generation with Deferred Positional Encoding](https://mlsanity.com/paper/2606.04302/) (2026)
- [MiniPIC: Flexible Position-Independent Caching in <100LOC](https://mlsanity.com/paper/2606.13126/) (2026)
- [HijackKV: New Threat in Position-Independent KV Cache Reuse](https://mlsanity.com/paper/2607.19957/) (2026)
- [Move the Query, Not the Cache: Characterizing Cross-Instance Latent Attention Redistribution Across GPU Fabrics](https://mlsanity.com/paper/2606.01502/) (2026)
- [CoMem: Reusing Transformer Depth across Queries with Persistent Intermediate Residuals](https://mlsanity.com/paper/2607.28263/) (2026)
- [ATFlash: Per-RoPE-Wavelength Attention Windows for Compute/Memory-Efficient LLM Inference](https://mlsanity.com/paper/2608.02947/) (2026)
- [Evaluating the accuracy of KV cache reuse techniques](https://mlsanity.com/paper/2609.31415/) (2026)
- [ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptation](https://mlsanity.com/paper/2609.36722/) (2026)
- [KV-Repurpose: A Testbed for Evaluating Non-Prefix KV-Cache Reuse](https://mlsanity.com/paper/2610.13060/) (2026)

## Abstract (arXiv)

Modern LLM applications such as deep-research assistants, coding agents, and Retrieval-Augmented Generation (RAG) systems, repeatedly process long prompt histories containing shared document or code chunks, creating significant pressure on the Key Value (KV) cache, which must operate within limited memory while sustaining high throughput and low latency. Prefix caching partially alleviates some of these costs by reusing KV cache for previously processed tokens, but limited by strict prefix matching. Position-independent caching (PIC) enables chunk-level reuse at arbitrary positions, but requires selective recomputation and positional-encoding (PE) adjustments. However, because these operations vary across queries, KV for the same chunk diverges across requests. Moreover, without page alignment, chunk KV layouts diverge in memory, preventing page sharing. These issues result in only modest HBM savings even when many requests reuse the same content. We present MEPIC, a memory-efficient PIC system that enables chunk KV reuse across positions, requests, and batches. MEPIC aligns chunk KV to paged storage, shifts recomputation from token- to block-level so only the first block is request-specific, removes positional encodings via Rotary Position Embedding (RoPE) fusion in the attention kernel, and makes remaining blocks fully shareable. These techniques eliminate most duplicate chunk KV in HBM, reducing usage by up to 2x over state-of-the-art PIC at comparable latency and accuracy, and up to 5x for long prompts, without any model changes.