# Caching Historical Embeddings in Conversational Search

arXiv: https://arxiv.org/abs/2211.14155 · HTML: https://mlsanity.com/paper/2211.14155/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2022.

## Built on by (most cited first, 6 of 6)

- [When Memory Becomes a Vulnerability: Towards Multi-turn Jailbreak Attacks against Text-to-Image Generation Systems](https://mlsanity.com/paper/2504.20376/) (2025)
- [QVCache: A Query-Aware Vector Cache](https://mlsanity.com/paper/2602.02057/) (2026)
- [PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling](https://mlsanity.com/paper/2609.09662/) (2026)
- [Leveraging Approximate Caching for Faster Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.05530/) (2025)
- [HaS: Accelerating RAG through Homology-Aware Speculative Retrieval](https://mlsanity.com/paper/2604.20452/) (2026)
- [Embedding Surgery: Localized Updates for Adaptive Ranking Correction in Dense Retrieval](https://mlsanity.com/paper/2609.05110/) (2026)

## Abstract (arXiv)

Rapid response, namely low latency, is fundamental in search applications; it is particularly so in interactive search sessions, such as those encountered in conversational settings. An observation with a potential to reduce latency asserts that conversational queries exhibit a temporal locality in the lists of documents retrieved. Motivated by this observation, we propose and evaluate a client-side document embedding cache, improving the responsiveness of conversational search systems. By leveraging state-of-the-art dense retrieval models to abstract document and query semantics, we cache the embeddings of documents retrieved for a topic introduced in the conversation, as they are likely relevant to successive queries. Our document embedding cache implements an efficient metric index, answering nearest-neighbor similarity queries by estimating the approximate result sets returned. We demonstrate the efficiency achieved using our cache via reproducible experiments based on TREC CAsT datasets, achieving a hit rate of up to 75% without degrading answer quality. Our achieved high cache hit rates significantly improve the responsiveness of conversational systems while likewise reducing the number of queries managed on the search back-end.