mlsanityData to 7 Oct 2026

IntentKV: Cross-Turn Intent-Aware KV Cache Pruning for Agent Inference

arXiv 2606.09916 ↗First posted 6 Jun 2026cs.LG cs.AIcited by 6

On the map: LLM agents and tool use → Software engineering coding agents

also in 1 more
Abstract (arXiv)

Multi-turn LLM agents fan short queries into long trajectories of tool calls, search results, and intermediate reasoning. Both KV memory and KV read bandwidth grow by orders of magnitude across a single trajectory, making the key-value (KV) cache, not parameter compute, the dominant serving bottleneck for long-horizon agents. We introduce IntentKV, learned KV pruning that keeps the base LLM frozen. IntentKV maintains a session-level QueryMemory of cross-turn intent, scores live history tokens with a memory-attention rule, and adds a zero-initialized residual head with cross-attention over current-query K-vectors. To stay composable with prefix caches, eviction is a slot-map redirection: dropped positions route to a sentinel dead slot while surviving K/V rows, RoPE phases, and slot identities stay in place. IntentKV matches the no-pruning full-cache baseline with almost no accuracy drop under tight KV budgets: at an 8k KV budget, mean peak request tokens drop 23.9% on Qwen3-8B and 30.7% on Qwen2.5-14B. On the 100 longest BCP queries that all methods complete on Qwen2.5-14B, IntentKV-8k further cuts worst-case peak request tokens from 92.3k to 20.5k, a 77.8% reduction, and worst-case raw KV reads from 411M to 31M, a 92.6% reduction.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 23 of its 28 references that are in the map. First posted on arXiv 6 Jun 2026.

Builds on

23 of 28 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning26 Feb
  2. 2025Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live4 Nov

Other references · 21

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  5. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  6. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  7. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  8. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
  9. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  10. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  11. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 487
  12. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 466
9 more
  1. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 428
  2. 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agentcited by 183
  3. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 158
  4. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLMcited by 155
  5. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 125
  6. 2024Long Context Compression with Activation Beaconcited by 101
  7. 2025Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distributioncited by 70
  8. 2025Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMscited by 18
  9. 2026LookaheadKV: Fast and Accurate KV Cache Eviction by Glimpsing into the Future without Generationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026MemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inferencecited by 3
  2. 2026PReM: Learning What to Preserve and When to Refresh for Context Compressioncited by 0
  3. 2026CommitKV: Lifecycle-Aware KV Cache Compression via Commit Transitions for Multi-Turn Agentscited by 0
  4. 2026Parser States Already Know: Structure-Conditioned KV Persistence for Structured Generationcited by 0
  5. 2026StepKV: Step-Aware KV Cache Compression for LLM Agentscited by 0
  6. 2026Persistent Context Graphs for Efficient Memory Compaction in LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Practical Online KV Cache Compaction for LLM Agents: An Empirical Study
  2. 2026AgentKV: Phase-Aware KV Eviction for Agentic LLMs
  3. 2026TriAxialKV: Toward Extreme Low-Precision KV-Cache Quantization for Agentic Inference Tasks
  4. 2026ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents
  5. 2026ActKV: Efficient LLM Agents through Action-Guided KV Cache Management
  6. 2026TokenPilot: Cache-Efficient Context Management for LLM Agents
  7. 2026AgentKVShift: Efficient KV Cache Reuse for Agentic Memory Systems
  8. 2026RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse