mlsanityData to 14 Oct 2026

IncDSI: Incrementally Updatable Document Retrieval

arXiv 2307.10323 ↗First posted 19 Jul 2023cs.IR cs.CL cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Differentiable Search Index is a recently proposed paradigm for document retrieval, that encodes information about a corpus of documents within the parameters of a neural network and directly maps queries to corresponding documents. These models have achieved state-of-the-art performances for document retrieval across many benchmarks. These kinds of models have a significant limitation: it is not easy to add new documents after a model is trained. We propose IncDSI, a method to add documents in real time (about 20-50ms per document), without retraining the model on the entire dataset (or even parts thereof). Instead we formulate the addition of documents as a constrained optimization problem that makes minimal changes to the network parameters. Although orders of magnitude faster, our approach is competitive with re-training the model on the whole dataset and enables the development of document retrieval systems that can be updated with new information in real-time. Our code for IncDSI is available at https://github.com/varshakishore/IncDSI.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 28 of its 44 references that are in the map. First posted on arXiv 19 Jul 2023.

Builds on

28 of 44 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  4. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  5. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  6. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  7. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  8. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  9. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  10. 2018Experience Replay for Continual Learningcited by 1,095
  11. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  12. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
16 more
  1. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 678
  2. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 521
  3. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 413
  4. 2022Transformer Memory as a Differentiable Search Indexcited by 247
  5. 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
  6. 2018Selective Experience Replay for Lifelong Learningcited by 241
  7. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 214
  8. 2019Multi-Stage Document Ranking with BERTcited by 201
  9. 2018Memory Efficient Experience Replay for Streaming Learningcited by 128
  10. 2017A Deep Relevance Matching Model for Ad-hoc Retrievalcited by 126
  11. 2022A Neural Corpus Indexer for Document Retrievalcited by 111
  12. 2022InPars: Data Augmentation for Information Retrieval using Large Language Modelscited by 62
  13. 2017Neural Ranking Models with Weak Supervisioncited by 61
  14. 2022Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generationcited by 48
  15. 2022DSI++: Updating Transformer Memory with New Documentscited by 35
  16. 2022Ultron: An Ultimate Retriever on Corpus with a Model-based Indexercited by 31

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 3; 2026: 2.

12 papers.

  1. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  2. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 83
  3. 2024ToolGen: Unified Tool Retrieval and Calling via Generationcited by 74
  4. 2023Continual Learning: Applications and the Road Forwardcited by 59
  5. 2024C-RAG: Certified Generation Risks for Retrieval-Augmented Language Modelscited by 27
  6. 2024Unifying Generative and Dense Retrieval for Sequential Recommendationcited by 18
  7. 2025Query Drift Compensation: Enabling Compatibility in Continual Learning of Retrieval Embedding Modelscited by 5
  8. 2024Generative Retrieval with Few-shot Indexingcited by 3
  9. 2025LoSemB: Logic-Guided Semantic Bridging for Inductive Tool Retrievalcited by 3
  10. 2024PromptDSI: Prompt-based Rehearsal-free Continual Learning for Document Retrievalcited by 1
  11. 2026ICICLE: Expanding Retrieval with In-Context Documentscited by 0
  12. 2026The Crowded Embedding Space: A Mean-Field Mechanism for Emergent Marginalization in Retrieval-Augmented Agentscited by 0