# CREAM: Continual Retrieval on Dynamic Streaming Corpora with Adaptive Soft Memory

arXiv: https://arxiv.org/abs/2601.02708 · HTML: https://mlsanity.com/paper/2601.02708/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 30 references that are in the map.
- First posted on arXiv 6 Jan 2026.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019)
- [ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction](https://mlsanity.com/paper/2112.01488/) (2021)
- [Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models](https://mlsanity.com/paper/2510.04618/) (2025)
- [Online Continual Learning with Maximally Interfered Retrieval](https://mlsanity.com/paper/1908.04742/) (2019)
- [Transformer Memory as a Differentiable Search Index](https://mlsanity.com/paper/2202.06991/) (2022)
- [Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling](https://mlsanity.com/paper/2104.06967/) (2021)
- [AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges](https://mlsanity.com/paper/2505.10468/) (2025)
- [Online Coreset Selection for Rehearsal-based Continual Learning](https://mlsanity.com/paper/2106.01085/) (2021)
- [Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipeline](https://mlsanity.com/paper/2101.08751/) (2021)
- [Continual Learning for Generative Retrieval over Dynamic Corpora](https://mlsanity.com/paper/2308.14968/) (2023)
- [MS-Shift: An Analysis of MS MARCO Distribution Shifts on Neural Retrieval](https://mlsanity.com/paper/2205.02870/) (2022)
- [L^2R: Lifelong Learning for First-stage Retrieval with Backward-Compatible Representations](https://mlsanity.com/paper/2308.11512/) (2023)
- [PDSum: Prototype-driven Continuous Summarization of Evolving Multi-document Sets Stream](https://mlsanity.com/paper/2302.05550/) (2023)
- [Unsupervised Story Discovery from Continuous News Streams via Scalable Thematic Embedding](https://mlsanity.com/paper/2304.04099/) (2023)
- [SCStory: Self-supervised and Continual Online Story Discovery](https://mlsanity.com/paper/2312.03725/) (2023)
- [Continual Learning of Long Topic Sequences in Neural Information Retrieval](https://mlsanity.com/paper/2201.03356/) (2022)

## Built on by (most cited first, 1 of 1)

- [ICICLE: Expanding Retrieval with In-Context Documents](https://mlsanity.com/paper/2605.26902/) (2026)

## Abstract (arXiv)

Information retrieval (IR) in dynamic data streams is a crucial task, as shifts in data distribution degrade the performance of AI-powered IR systems. To mitigate this issue, memory-based continual learning has been widely adopted for IR. However, existing methods rely on a fixed set of queries with ground-truth documents, which limits generalization to unseen data, making them impractical for real-world applications. To enable more effective learning with unseen topics of a new corpus without ground-truth labels, we propose CREAM, a self-supervised framework for memory-based continual retrieval. CREAM captures the evolving semantics of streaming queries and documents into dynamically structured soft memory and leverages it to adapt to both seen and unseen topics in an unsupervised setting. We realize this through three key techniques: fine-grained similarity estimation, regularized cluster prototyping, and stratified coreset sampling. Experiments on two benchmark datasets demonstrate that CREAM exhibits superior adaptability and retrieval accuracy, outperforming the strongest method in a label-free setting by 27.79% in Success@5 and 44.5% in Recall@10 on average, and achieving performance comparable to or even exceeding that of supervised methods.