# Positions Are Not Facts: The Mismatch Between KV Caches and Memory

arXiv: https://arxiv.org/abs/2609.33759 · HTML: https://mlsanity.com/paper/2609.33759/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving.
- Builds on 34 of its 72 references that are in the map.
- First posted on arXiv 27 Sep 2026.

## Placed in

- Efficient LLM inference and serving / [KV cache optimization](https://mlsanity.com/inference/kv-cache-optimization/)

## Builds on (most cited first, 30 of 34)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Zep: A Temporal Knowledge Graph Architecture for Agent Memory](https://mlsanity.com/paper/2501.13956/) (2025)
- [Prompt Cache: Modular Attention Reuse for Low-Latency Inference](https://mlsanity.com/paper/2311.04934/) (2023)
- [MemOS: A Memory OS for AI System](https://mlsanity.com/paper/2507.03724/) (2025)
- [LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference](https://mlsanity.com/paper/2510.09665/) (2025)
- [MEMORYLLM: Towards Self-Updatable Large Language Models](https://mlsanity.com/paper/2402.04624/) (2024)
- [Cartridges: Lightweight and general-purpose long context representations via self-study](https://mlsanity.com/paper/2506.06266/) (2025)
- [KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse](https://mlsanity.com/paper/2502.16002/) (2025)
- [Cache-to-Cache: Direct Semantic Communication Between Large Language Models](https://mlsanity.com/paper/2510.03215/) (2025)
- [STALE: Can LLM Agents Know When Their Memories Are No Longer Valid?](https://mlsanity.com/paper/2605.06527/) (2026)
- [DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving](https://mlsanity.com/paper/2411.02820/) (2024)
- [Unable to Forget: Proactive Interference Reveals Working Memory Limits in LLMs Beyond Context Length](https://mlsanity.com/paper/2506.08184/) (2025)
- [Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks](https://mlsanity.com/paper/2601.06007/) (2026)
- [KVComm: Enabling Efficient LLM Communication through Selective KV Sharing](https://mlsanity.com/paper/2510.03346/) (2025)
- [MEMENTO: Teaching LLMs to Manage Their Own Context](https://mlsanity.com/paper/2604.09852/) (2026)
- [Analysing the Residual Stream of Language Models Under Knowledge Conflicts](https://mlsanity.com/paper/2410.16090/) (2024)
- [Latent Space Communication via K-V Cache Alignment](https://mlsanity.com/paper/2601.06123/) (2026)
- [Do Large Language Models Need a Content Delivery Network?](https://mlsanity.com/paper/2409.13761/) (2024)
- [Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse](https://mlsanity.com/paper/2608.03893/) (2026)
- [Learning to Evict from Key-Value Cache](https://mlsanity.com/paper/2602.10238/) (2026)
- [Activated LoRA: Fine-tuned LLMs for Intrinsics](https://mlsanity.com/paper/2504.12397/) (2025)
- [SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning](https://mlsanity.com/paper/2602.22603/) (2026)
- [Learning to Forget: Sleep-Inspired Memory Consolidation for Resolving Proactive Interference in Large Language Models](https://mlsanity.com/paper/2603.14517/) (2026)
- [Predicting Future Utility: Global Combinatorial Optimization for Task-Agnostic KV Cache Eviction](https://mlsanity.com/paper/2602.08585/) (2026)
- [Do Language Models Track Entities Across State Changes?](https://mlsanity.com/paper/2605.30233/) (2026)
- [KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing](https://mlsanity.com/paper/2606.17034/) (2026)
- [Models Take Notes at Prefill: KV Cache Can Be Editable and Composable](https://mlsanity.com/paper/2606.17107/) (2026)
- [KEEP: A KV-Cache-Centric Memory Management System for Efficient Embodied Planning](https://mlsanity.com/paper/2602.23592/) (2026)
- [MINTEval: Evaluating Memory under Multi-Target Interference in Long-Horizon Agent Systems](https://mlsanity.com/paper/2605.18565/) (2026)
- [Self-Pruned Key-Value Attention: Learning When to Write by Predicting Future Utility](https://mlsanity.com/paper/2605.14037/) (2026)

## Abstract (arXiv)

When a fact changes, how should a language model update the history stored in its key-value (KV) cache? Hiding the old record is cheap, but it may still contain needed details or answer questions about the past. We compare hiding whole records, hiding only replaced values, and deleting old text and recomputing the cache. In a controlled quantity task, masking makes all eight models prefer the new value more strongly, yet six lose complete answers through unit errors or failure to stop; keeping the unit preserves all current answers. Later states also retain useful information from earlier records: on multi-hop updates, rebuilding these states at unchanged positions lowers historical accuracy by 20-41 percentage points, whereas moving the existing states has little effect. Keeping object dependencies and unchanged revision passages prevents many losses. Recognition is a separate challenge. Learned readouts recover distinctions missed by fixed cache similarities on synthetic record pairs. On natural text, text-detector-selected masks show no clear advantage over random masks at the same rate in 14 same-model detector-generator comparisons. Query-dependent access can avoid some losses, with additional storage or access costs. These findings identify what must be preserved beyond the replaced value when using a KV cache as updatable memory.