mlsanityData to 7 Oct 2026

Why Agent Caching Fails and How to Fix It: Structured Intent Canonicalization with Few-Shot Learning

arXiv 2602.18922 ↗First posted 21 Feb 2026cs.CL cs.AI cs.LGcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Personal AI agents incur substantial cost via repeated LLM calls. We show existing caching methods fail: GPTCache achieves 37.9% accuracy on real benchmarks; APC achieves 0-12%. The root cause is optimizing for the wrong property -- cache effectiveness requires key consistency and precision, not classification accuracy. We observe cache-key evaluation reduces to clustering evaluation and apply V-measure decomposition to separate these on n=8,682 points across MASSIVE, BANKING77, CLINC150, and NyayaBench v2, our new 8,514-entry multilingual agentic dataset (528 intents, 20 W5H2 classes, 63 languages). We introduce W5H2, a structured intent decomposition framework. Using SetFit with 8 examples per class, W5H2 achieves 91.1%+/-1.7% on MASSIVE in ~2ms -- vs 37.9% for GPTCache and 68.8% for a 20B-parameter LLM at 3,447ms. On NyayaBench v2 (20 classes), SetFit achieves 55.3%, with cross-lingual transfer across 30 languages. Our five-tier cascade handles 85% of interactions locally, projecting 97.5% cost reduction. We provide risk-controlled selective prediction guarantees via RCPS with nine bound families.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 19 of its 26 references that are in the map. First posted on arXiv 21 Feb 2026.

Builds on

19 of 26 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026SemanticALLI: Caching Reasoning, Not Just Responses, in Agentic Systems22 Jan
  2. 2025Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agents17 Jun

Other references · 17

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  5. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  6. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  7. 2017Selective Classification for Deep Neural Networkscited by 799
  8. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 419
  9. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 378
  10. 2021Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Controlcited by 256
  11. 2019BERT for Joint Intent Classification and Slot Fillingcited by 195
  12. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 142
5 more
  1. 2022Efficient Few-Shot Learning Without Promptscited by 135
  2. 2024RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generationcited by 97
  3. 2020DIET: Lightweight Language Understanding for Dialogue Systemscited by 35
  4. 2025vCache: Verified Semantic Prompt Cachingcited by 15
  5. 2025Advancing Semantic Caching for LLMs with Domain-Specific Embeddings and Synthetic Datacited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks
  2. 2026AgenticCache: Cache-Driven Asynchronous Planning for Embodied AI Agents
  3. 2026MemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inference
  4. 2026Learning Agent Execution for KV-Cache Management in Agentic Serving
  5. 2026Irminsul: MLA-Native Position-Independent Caching for Agentic LLM Serving
  6. 2026RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse
  7. 2026On Device Agentic Operation Caches -- Classifier-Centric NL-to-Action Generation
  8. 2026IntentKV: Cross-Turn Intent-Aware KV Cache Pruning for Agent Inference