mlsanityData to 7 Oct 2026

SemPIC: Learning Semantic Position-Independent KV Caches

arXiv 2607.28069 ↗First posted 30 Jul 2026cs.AIcited by 2

On the map: LLM agents and tool use → Software engineering coding agents

also in 1 more
Abstract (arXiv)

Long-context retrieval and agentic workloads repeatedly reuse the same documents under changing instructions, histories, and document orders. Prefix caching cannot exploit this reuse, while position-independent caching (PIC) remains unreliable because independently compiled KV states lack the future context in which they will be consumed. Our diagnostics show that a learned boundary-conditioned baseline sharply reduces attention deviation near reusable-block boundaries but leaves interior and task-level residuals, motivating adaptation of the document representation itself. We present SemPIC, which trains a LoRA-enabled Writer to compile native per-layer document KVs through behavioral distillation while retaining the pretrained decoder as an unchanged Reader. Adaptation is confined to offline cache construction, preserving the standard KV interface and cache-hit decoding path. We further introduce KV Gradient Checkpointing, which reduces peak training memory without severing gradients through cached KVs. Across three models and four tasks, SemPIC raises mean micro-F1 over KV Packet from 0.53 to 0.60, approaching Full Recompute at 0.62. Code: https://github.com/jn12-29/SemPIC

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 25 of its 33 references that are in the map. First posted on arXiv 30 Jul 2026.

Builds on

25 of 33 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  8. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  9. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  10. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  11. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  12. 2016Training Deep Nets with Sublinear Memory Costcited by 825
13 more
  1. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  2. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headscited by 283
  3. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  4. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  5. 2024When Attention Sink Emerges in Language Models: An Empirical Viewcited by 200
  6. 2024ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identificationcited by 86
  7. 2024TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Textcited by 60
  8. 2025KVLink: Accelerating Large Language Models via Efficient KV Cache Reusecited by 60
  9. 2025CacheClip: Accelerating RAG with Effective KV Cache Reusecited by 14
  10. 2026KV Packet: Recomputation-Free Context-Independent KV Caching for LLMscited by 10
  11. 2025A3: Attention-Aware Accurate KV Cache Fusion for Fast Large Language Model Servingcited by 7
  12. 2026You Need an Encoder for Native Position-Independent Cachingcited by 6
  13. 2026QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Servingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptationcited by 0
  2. 2026Cache the Encoder Within:Compact, Reusable Memory across LLM Queriescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026OVAL: Output-Aware Local Page Bases for KV Cache Retrieval
  2. 2026KV-Kaizen: Learning Context-Adaptive Cache Compression Choices
  3. 2026Behavior-Preserving KV Cache Compression
  4. 2026PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression
  5. 2025Task-KV: Task-aware KV Cache Optimization via Semantic Differentiation of Attention Heads
  6. 2026Contiguity, Not Importance: Budgeted Repair of Stale KV Caches After Document Edits
  7. 2026Rethinking LoRA Memory Through the Lens of KV Cache Compression
  8. 2026More Than a Quick Glance: Overcoming the Greedy Bias in KV-Cache Compression