mlsanityData to 14 Oct 2026

ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents

arXiv 2608.19662 ↗First posted 20 Aug 2026cs.CLcited by 3

On the map: LLM agents and tool use → Software engineering coding agents

also in 1 more
Abstract (arXiv)

Agentic language models repeatedly encode tool and skill schemas that recur across requests in different combinations and orders, preventing standard prefix caching from reusing their key--value (KV) states. We introduce ReCache, a framework for independently caching resource representations while reducing their inference-time computational and memory overhead. Resource-wise attention removes cross-resource interactions and assigns resource-local positions, producing composition-invariant KV blocks. ReCache then restricts resource visibility to contribution-selected layer--KV-head-group routes and retains only invocation-critical fields through structural and semantic pruning. We evaluate ReCache on a benchmark assembled from seven public tool- and skill-use datasets, including resource-disjoint tests. Resource-wise attention matches dense invocation performance (82.3% versus 82.4% Inv-F1) while providing a 3.655× time-to-first-token speedup. The complete framework reduces allocated KV-tensor memory by 92.43% and accelerates attention by 1.423×. These results show that separating reusable schema encoding from selective resource access substantially reduces agentic inference costs with limited effectiveness loss. The code is available at https://github.com/EIT-NLP/ReCache.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 41 of its 53 references that are in the map. First posted on arXiv 20 Aug 2026.

Builds on

41 of 53 references.

References · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  6. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  7. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  8. 2018Self-Attention with Relative Position Representationscited by 1,120
  9. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  10. 2019Are Sixteen Heads Really Better than One?cited by 816
  11. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 591
  12. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 439
29 more
  1. 2023Learning to Compress Prompts with Gist Tokenscited by 390
  2. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 383
  3. 2023Adapting Language Models to Compress Contextscited by 356
  4. 2023The Impact of Positional Encoding on Length Generalization in Transformerscited by 327
  5. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 315
  6. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 313
  7. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
  8. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 240
  9. 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 232
  10. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 223
  11. 2024ToolACE: Winning the Points of LLM Function Callingcited by 216
  12. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  13. 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasetscited by 198
  14. 2022Transformer Language Models without Positional Encodings Still Learn Positional Informationcited by 150
  15. 2023RestGPT: Connecting Large Language Models with Real-World RESTful APIscited by 135
  16. 2026SoK: Agentic Skills -- Beyond Tool Use in LLM Agentscited by 130
  17. 2024Long Context Compression with Activation Beaconcited by 104
  18. 2024ToolGen: Unified Tool Retrieval and Calling via Generationcited by 74
  19. 2025TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP Environmentscited by 69
  20. 2025KVLink: Accelerating Large Language Models via Efficient KV Cache Reusecited by 66
  21. 2023Stateful Large Language Model Serving with Pensievecited by 61
  22. 2026SkillRouter: Skill Routing for LLM Agents at Scalecited by 61
  23. 2025ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inferencecited by 60
  24. 2024PromptReps: Prompting Large Language Models to Generate Dense and Sparse Representations for Zero-Shot Document Retrievalcited by 30
  25. 2025ToolMind Technical Report: A Large-Scale, Reasoning-Enhanced Tool-Use Datasetcited by 30
  26. 2026ToolOmni: Enabling Open-World Tool Use via Agentic learning with Proactive Retrieval and Grounded Executioncited by 7
  27. 2026SkillZip: Evaluation-Free Skill Compression for Self-Evolving Agents by Discovering Reusable Structurecited by 4
  28. 2026DepthKV: Layer-Dependent KV Cache Pruning for Long-Context LLM Inferencecited by 3
  29. 2026Shallow Prefill, Deep Decoding: Efficient Long-Context Inference via Layer-Asymmetric KV Visibilitycited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026What It Costs to Compose, Rebuild, and Correct Precomputed Memorycited by 1
  2. 2026Memory as a cache: Exact context reuse and deletion by constructioncited by 0
  3. 2026ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inference
  2. 2026ToolAtlas: Learning Once, Reusing Everywhere with Tool-Side Memory
  3. 2025KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows
  4. 2026TVCACHE: A Stateful Tool-Value Cache for Post-Training LLM Agents
  5. 2026Irminsul: MLA-Native Position-Independent Caching for Agentic LLM Serving
  6. 2026Learning Agent Execution for KV-Cache Management in Agentic Serving
  7. 2026RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse
  8. 2025KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems