# Continual Learning for Generative Retrieval over Dynamic Corpora

arXiv: https://arxiv.org/abs/2308.14968 · HTML: https://mlsanity.com/paper/2308.14968/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Aug 2023.

## Built on by (most cited first, 25 of 25)

- [From Matching to Generation: A Survey on Generative Information Retrieval](https://mlsanity.com/paper/2404.14851/) (2024)
- [ToolGen: Unified Tool Retrieval and Calling via Generation](https://mlsanity.com/paper/2410.03439/) (2024)
- [Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding](https://mlsanity.com/paper/2404.14600/) (2024)
- [Unifying Generative and Dense Retrieval for Sequential Recommendation](https://mlsanity.com/paper/2411.18814/) (2024)
- [A Survey of Generative Search and Recommendation in the Era of Large Language Models](https://mlsanity.com/paper/2404.16924/) (2024)
- [Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2407.06992/) (2024)
- [CorpusBrain++: A Continual Generative Pre-Training Framework for Knowledge-Intensive Language Tasks](https://mlsanity.com/paper/2402.16767/) (2024)
- [Lightweight and Direct Document Relevance Optimization for Generative Information Retrieval](https://mlsanity.com/paper/2504.05181/) (2025)
- [Generative Retrieval as Multi-Vector Dense Retrieval](https://mlsanity.com/paper/2404.00684/) (2024)
- [Generative Information Retrieval Evaluation](https://mlsanity.com/paper/2404.08137/) (2024)
- [RAGVA: Engineering Retrieval Augmented Generation-based Virtual Assistants in Practice](https://mlsanity.com/paper/2502.14930/) (2025)
- [A Survey of Generative Information Retrieval](https://mlsanity.com/paper/2406.01197/) (2024)
- [MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora](https://mlsanity.com/paper/2507.09924/) (2025)
- [Generative Retrieval with Few-shot Indexing](https://mlsanity.com/paper/2408.02152/) (2024)
- [When Should Dense Retrievers Be Updated in Evolving Corpora? Detecting Out-of-Distribution Corpora Using GradNormIR](https://mlsanity.com/paper/2506.01877/) (2025)
- [RIGHT: Retrieval-augmented Generation for Mainstream Hashtag Recommendation](https://mlsanity.com/paper/2312.10466/) (2023)
- [Bootstrapped Pre-training with Dynamic Identifier Prediction for Generative Retrieval](https://mlsanity.com/paper/2407.11504/) (2024)
- [Know Where to Go: Make LLM a Relevant, Responsible, and Trustworthy Searcher](https://mlsanity.com/paper/2310.12443/) (2023)
- [PromptDSI: Prompt-based Rehearsal-free Continual Learning for Document Retrieval](https://mlsanity.com/paper/2406.12593/) (2024)
- [DRAMA: Unifying Data Retrieval and Analysis for Open-Domain Analytic Queries](https://mlsanity.com/paper/2510.27238/) (2025)
- [CREAM: Continual Retrieval on Dynamic Streaming Corpora with Adaptive Soft Memory](https://mlsanity.com/paper/2601.02708/) (2026)
- [A Parametric Memory Head for Continual Generative Retrieval](https://mlsanity.com/paper/2604.23388/) (2026)
- [SODIUM: From Open Web Data to Queryable Databases](https://mlsanity.com/paper/2603.18447/) (2026)
- [ICICLE: Expanding Retrieval with In-Context Documents](https://mlsanity.com/paper/2605.26902/) (2026)
- [Generative Universal Multimodal Retrieval with Dual-role Identifiers](https://mlsanity.com/paper/2608.12987/) (2026)

## Abstract (arXiv)

Generative retrieval (GR) directly predicts the identifiers of relevant documents (i.e., docids) based on a parametric model. It has achieved solid performance on many ad-hoc retrieval tasks. So far, these tasks have assumed a static document collection. In many practical scenarios, however, document collections are dynamic, where new documents are continuously added to the corpus. The ability to incrementally index new documents while preserving the ability to answer queries with both previously and newly indexed relevant documents is vital to applying GR models. In this paper, we address this practical continual learning problem for GR. We put forward a novel Continual-LEarner for generatiVE Retrieval (CLEVER) model and make two major contributions to continual learning for GR: (i) To encode new documents into docids with low computational cost, we present Incremental Product Quantization, which updates a partial quantization codebook according to two adaptive thresholds; and (ii) To memorize new documents for querying without forgetting previous knowledge, we propose a memory-augmented learning mechanism, to form meaningful connections between old and new documents. Empirical results demonstrate the effectiveness and efficiency of the proposed model.