# LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs

arXiv: https://arxiv.org/abs/2608.11231 · HTML: https://mlsanity.com/paper/2608.11231/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving.
- Builds on 25 of its 32 references that are in the map.
- First posted on arXiv 31 Jul 2026.

## Placed in

- Efficient LLM inference and serving / [KV cache optimization](https://mlsanity.com/inference/kv-cache-optimization/)

## Builds on (most cited first, 25 of 25)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [SGLang: Efficient Execution of Structured Language Model Programs](https://mlsanity.com/paper/2312.07104/) (2023)
- [Gated Delta Networks: Improving Mamba2 with Delta Rule](https://mlsanity.com/paper/2412.06464/) (2024)
- [Prompt Cache: Modular Attention Reuse for Low-Latency Inference](https://mlsanity.com/paper/2311.04934/) (2023)
- [CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion](https://mlsanity.com/paper/2405.16444/) (2024)
- [KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse](https://mlsanity.com/paper/2502.16002/) (2025)
- [EPIC: Efficient Position-Independent Caching for Serving Large Language Models](https://mlsanity.com/paper/2410.15332/) (2024)
- [AI Flow: Perspectives, Scenarios, and Approaches](https://mlsanity.com/paper/2506.12479/) (2025)
- [Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.15734/) (2025)
- [Marconi: Prefix Caching for the Era of Hybrid LLMs](https://mlsanity.com/paper/2411.19379/) (2024)
- [KVShare: An LLM Service System with Efficient and Effective Multi-Tenant KV Cache Reuse](https://mlsanity.com/paper/2503.16525/) (2025)
- [CacheClip: Accelerating RAG with Effective KV Cache Reuse](https://mlsanity.com/paper/2510.10129/) (2025)
- [ProphetKV: User-Query-Driven Selective Recomputation for Efficient KV Cache Reuse in Retrieval-Augmented Generation](https://mlsanity.com/paper/2602.02579/) (2026)
- [MEPIC: Memory Efficient Position Independent Caching for LLM Serving](https://mlsanity.com/paper/2512.16822/) (2025)
- [HYPIC: Accelerating Hybrid-Attention LLM Serving with Position-Independent Caching](https://mlsanity.com/paper/2607.01299/) (2026)
- [You Need an Encoder for Native Position-Independent Caching](https://mlsanity.com/paper/2602.01519/) (2026)
- [Sparse Prefix Caching for Hybrid and Recurrent LLM Serving](https://mlsanity.com/paper/2605.05219/) (2026)
- [Rethinking State Tracking in Recurrent Models Through Error Control Dynamics](https://mlsanity.com/paper/2605.07755/) (2026)
- [Computation-Bandwidth-Memory Trade-offs: A Unified Paradigm for AI Infrastructure](https://mlsanity.com/paper/2601.11577/) (2025)
- [Irminsul: MLA-Native Position-Independent Caching for Agentic LLM Serving](https://mlsanity.com/paper/2605.05696/) (2026)
- [Compiler-First State Space Duality and Portable O(1) Autoregressive Caching for Inference](https://mlsanity.com/paper/2603.09555/) (2026)

## Built on by (most cited first, 3 of 3)

- [Tail-Replay: Escaping the Curse of Linear Attention in Prefix Caching for Hybrid LLMs](https://mlsanity.com/paper/2608.30310/) (2026)
- [Memory as a cache: Exact context reuse and deletion by construction](https://mlsanity.com/paper/2609.32395/) (2026)
- [Just Let Linear States Forget the Distant Past: Prefix Caching via Suffix Replay for Hybrid LLMs](https://mlsanity.com/paper/2609.33477/) (2026)

## Abstract (arXiv)

LLM serving is increasingly accelerated by position-independent caching (PIC). Existing PIC methods, however, are built for full-attention models, where a token-indexed KV cache underlies its core operations: matching reusable token chunks, concatenating their KV entries, and selectively recomputing a few tokens to restore cross-chunk context. Hybrid LLMs break these primitives---they replace most attention layers with linear recurrences that expose only a fixed-size state, leaving no token-indexed KV to concatenate or to locally repair. This raises a natural question: can PIC benefit hybrid models, and what would it take? We present LinearKV, a training-free hybrid-PIC framework. Its key insight is a decoupled initialization: each linear layer maps its K matched local states to a single initial state, while full-attention layers concatenate their KV as before. LinearKV is therefore compatible with existing PIC methods, reusing their token selection and recomputation as-is. Under this framework, we find that a single cached state suffices as the linear layer's initializer. The algebraically principled alternative---composing all K cached states into the exact full-prefix state, as concurrent work HYPIC does---is unnecessary and, on some architectures, even harmful. We compare the two across three hybrid models and three PIC selectors. On the two GDN models the two tie, both recovering most of full quality (up to 92%); on the Mamba-2 model, exact composition instead collapses under every selector---under EPIC, for instance, it recovers only 46.6% of full quality, versus 86.8% for a single cached block initializer. A single state initializer is also cheaper, cutting time-to-first-token to 0.46× full prefill versus a further 5--17% overhead for exact composition; results hold across LongBench QA and RULER at 8K--32K.