# ContextCache: Context-Aware Semantic Cache for Multi-Turn Queries in Large Language Models

arXiv: https://arxiv.org/abs/2506.22791 · HTML: https://mlsanity.com/paper/2506.22791/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 7 references that are in the map.
- First posted on arXiv 28 Jun 2025.

## Builds on (most cited first, 4 of 4)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Long Range Arena: A Benchmark for Efficient Transformers](https://mlsanity.com/paper/2011.04006/) (2020)

## Built on by (most cited first, 7 of 7)

- [Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks](https://mlsanity.com/paper/2601.06007/) (2026)
- [QUASAR: A Universal Autonomous System for Atomistic Simulation and a Benchmark of Its Capabilities](https://mlsanity.com/paper/2602.00185/) (2026)
- [From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching](https://mlsanity.com/paper/2601.23088/) (2026)
- [Grounded Cache Routing for Retrieval-Augmented Generation: When Is It Safe to Reuse an Answer?](https://mlsanity.com/paper/2605.27494/) (2026)
- [Continuous Semantic Caching for Low-Cost LLM Serving](https://mlsanity.com/paper/2604.20021/) (2026)
- [Which Eviction Policy Should an LLM Cache Use? A Systematic Study Across Workloads, Capacities, and Encoders](https://mlsanity.com/paper/2608.20280/) (2026)
- [AgentDecarbonizer: Carbon-Aware Execution for AI Agents](https://mlsanity.com/paper/2608.20566/) (2026)

## Abstract (arXiv)

Semantic caching significantly reduces computational costs and improves efficiency by storing and reusing large language model (LLM) responses. However, existing systems rely primarily on matching individual queries, lacking awareness of multi-turn dialogue contexts, which leads to incorrect cache hits when similar queries appear in different conversational settings. This demonstration introduces ContextCache, a context-aware semantic caching system for multi-turn dialogues. ContextCache employs a two-stage retrieval architecture that first executes vector-based retrieval on the current query to identify potential matches and then integrates current and historical dialogue representations through self-attention mechanisms for precise contextual matching. Evaluation of real-world conversations shows that ContextCache improves precision and recall compared to existing methods. Additionally, cached responses exhibit approximately 10 times lower latency than direct LLM invocation, enabling significant computational cost reductions for LLM conversational applications.