mlsanityData to 7 Oct 2026

Irminsul: MLA-Native Position-Independent Caching for Agentic LLM Serving

arXiv 2605.05696 ↗First posted 7 May 2026cs.DC cs.AI cs.LGcited by 3

On the map: LLM agents and tool use → Software engineering coding agents

also in 1 more
Abstract (arXiv)

Agentic LLM workloads put bit-identical tokens at shifted positions every turn, voiding prefix caches at the first byte of divergence. Operators report cache-hit regressions ranging from moderate slowdowns to severe TTFT spikes of 10-16s on unchanged content. Prior position-independent caching systems correct RoPE on the full dK-dimensional key, an architectural cost imposed by GQA, not by caching itself. Multi-Head Latent Attention, deployed at scale in DeepSeek-V2/V3/R1, Kimi-K2/Moonlight, GLM-5, and Mistral Large 3, factors each KV row into a position-free cKV and a 64-dim kr correctable in closed form; this structure motivates content-addressed caching as a natural fit rather than a GQA workaround. We present Irminsul, which extends SGLang's radix cache with content-hash keying over CDC-chunked segments and a δ-rotation rule for kr. We evaluate three native MLA-MoE deployments - DeepSeek-V2-Lite (16B/2.4B), Kimi Moonlight-16B-A3B, and JoyAI-Flash (48B/3B) - with output-consistency on all three and recovery measured on the two endpoints; Irminsul recovers up to ~83% of prompt tokens above exact-prefix on agentic traffic while delivering 63% prefill energy savings per cache hit. We argue that content-addressed caching belongs in the serving stack as a first-class primitive, not a retrofit over prefix matching.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 18 of its 32 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

18 of 32 references.

References · 18

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  4. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  5. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  6. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  7. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 483
  8. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  9. 2025The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Executioncited by 73
  10. 2025On the Emergence of Position Bias in Transformerscited by 57
  11. 2024EPIC: Efficient Position-Independent Caching for Serving Large Language Modelscited by 45
  12. 2024Marconi: Prefix Caching for the Era of Hybrid LLMscited by 29
6 more
  1. 2025MEPIC: Memory Efficient Position Independent Caching for LLM Servingcited by 10
  2. 2026Attention Sinks Are Provably Necessary in Softmax Transformers: Evidence from Trigger-Conditional Taskscited by 10
  3. 2026KV Packet: Recomputation-Free Context-Independent KV Caching for LLMscited by 10
  4. 2025SemShareKV: Efficient KVCache Sharing for Semantically Similar Prompts via Token-Level LSH Matchingcited by 6
  5. 2026SnapMLA: Efficient Long-Context MLA Decoding via Hardware-Aware FP8 Quantized Pipeliningcited by 3
  6. 2026What Makes Position Zero Special? A Mechanistic Study of Position Zero Attention Sinks in LLMscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Leyline: KV Cache Directives for Agentic Inferencecited by 5
  2. 2026LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMscited by 3
  3. 2026VestigeKV: The NoPE-MLA KV Cache Carries Its Own Sparse-Attention Signal in a Vestigial Branchcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents
  2. 2026MemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inference
  3. 2026AgentKV: Phase-Aware KV Eviction for Agentic LLMs
  4. 2026TokenPilot: Cache-Efficient Context Management for LLM Agents
  5. 2026IntentKV: Cross-Turn Intent-Aware KV Cache Pruning for Agent Inference
  6. 2026MiniPIC: Flexible Position-Independent Caching in <100LOC
  7. 2026HYPIC: Accelerating Hybrid-Attention LLM Serving with Position-Independent Caching
  8. 2026Practical Online KV Cache Compaction for LLM Agents: An Empirical Study