Transactional Attention: Semantic Sponsorship for KV-Cache Retention
On the map: Efficient LLM inference and serving → KV cache optimization
Abstract (arXiv)
At K=16 tokens (0.4% of a 4K context), every existing KV-cache compression method achieves 0% on credential retrieval. The failure mode is dormant tokens: credentials, API keys, and configuration values that receive near-zero attention but become essential at generation time. Because these tokens lack the statistical signals that eviction policies rely on, no method based on attention scores, reconstruction loss, or learned retention gates retains them. We introduce Transactional Attention (TA), a sponsorship mechanism in which structural anchor patterns (e.g., "key:", "password:") protect adjacent value-bearing tokens from eviction. TA achieves 100% credential retrieval at K=16 where six baselines (H2O, TOVA, SnapKV, StreamingLLM, PyramidKV, DynamicKV) achieve 0%, and sustains 100% accuracy across 200 function-calling trials. TA-Fast, an attention-free variant, reduces memory overhead by 52% and is compatible with SDPA and FlashAttention. TA is orthogonal to existing compression methods and adds less than 1% latency overhead.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 21 of its 28 references that are in the map. First posted on arXiv 13 Apr 2026.
Builds on
21 of 28 references.
Closest ancestors · 10
Same direction, newest first
- 2026Understanding the Physics of Key-Value Cache Compression for LLMs through Attention Dynamics2 Mar
- 2025Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs3 Dec
- 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction29 May
- 2025RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression19 Feb
- 2025ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference1 Feb
- 2024DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMs19 Dec
- 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference16 Jul
- 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling4 Jun
- 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
- 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun
Other references · 11
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2016Gaussian Error Linear Units (GELUs)cited by 3,431
- 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
- 2024Transformers are Multi-State RNNscited by 120
- 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Modelscited by 52
- 2026SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoningcited by 10
- 2026StructKV: Preserving the Structural Skeleton for Scalable Long-Context Inferencecited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024RazorAttention: Efficient KV Cache Compression Through Retrieval Heads
- 2026Behavior-Preserving KV Cache Compression
- 2025CAOTE: KV Cache Selection for LLMs via Attention Output Error-Based Token Eviction
- 2026KV2: A Self-Refining KV Cache
- 2026Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction
- 2026More Than a Quick Glance: Overcoming the Greedy Bias in KV-Cache Compression
- 2024HashEvict: A Pre-Attention KV Cache Eviction Strategy using Locality-Sensitive Hashing
- 2026AnchorKV: Safety-Aware KV Cache Compression via Soft Penalty with a Refusal Anchor