mlsanityData to 7 Oct 2026

Exploring the Practicality of Generative Retrieval on Dynamic Corpora

arXiv 2305.18952 ↗First posted 27 May 2023cs.IR cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Benchmarking the performance of information retrieval (IR) is mostly conducted with a fixed set of documents (static corpora). However, in realistic scenarios, this is rarely the case and the documents to be retrieved are constantly updated and added. In this paper, we focus on Generative Retrievals (GR), which apply autoregressive language models to IR problems, and explore their adaptability and robustness in dynamic scenarios. We also conduct an extensive evaluation of computational and memory efficiency, crucial factors for real-world deployment of IR systems handling vast and ever-changing document collections. Our results on the StreamingQA benchmark demonstrate that GR is more adaptable to evolving knowledge (4-11%), robust in learning knowledge with temporal information, and efficient in terms of inference FLOPs (x2), indexing time (x6), and storage footprint (x4) compared to Dual Encoders (DE), which are commonly used in retrieval systems. Our paper highlights the potential of GR for future use in practical IR systems within dynamic environments.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 42 of its 65 references that are in the map. First posted on arXiv 27 May 2023.

Builds on

42 of 65 references.

References · 42

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  7. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  8. 2017Billion-scale similarity search with GPUscited by 2,477
  9. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  10. 2019Language Models as Knowledge Bases?cited by 1,922
  11. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  12. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
30 more
  1. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  2. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  3. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  4. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  5. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  6. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  7. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  8. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  9. 2020Autoregressive Entity Retrievalcited by 333
  10. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  11. 2022Transformer Memory as a Differentiable Search Indexcited by 242
  12. 2021Composable Sparse Fine-Tuning for Cross-Lingual Transfercited by 144
  13. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 123
  14. 2022Autoregressive Search Engines: Generating Substrings as Document Identifierscited by 112
  15. 2014Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)cited by 109
  16. 2022A Neural Corpus Indexer for Document Retrievalcited by 107
  17. 2021Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inferencecited by 96
  18. 2021A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniquescited by 69
  19. 2021Learning Passage Impacts for Inverted Indexescited by 68
  20. 2023Learning to Tokenize for Generative Retrievalcited by 66
  21. 2022Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generationcited by 47
  22. 2022Questions Are All You Need to Train a Dense Passage Retrievercited by 47
  23. 2023How Does Generative Retrieval Scale to Millions of Passages?cited by 47
  24. 2018End-to-End Retrieval in Continuous Spacecited by 46
  25. 2021Learning to Retrieve Passages without Supervisioncited by 46
  26. 2022CorpusBrain: Pre-train a Generative Retrieval Model for Knowledge-Intensive Language Taskscited by 36
  27. 2022DSI++: Updating Transformer Memory with New Documentscited by 35
  28. 2021Rethinking Search: Making Domain Experts out of Dilettantescited by 18
  29. 2022DynamicRetriever: A Pre-training Model-based IR System with Neither Sparse nor Dense Indexcited by 12
  30. 2023LongEval-Retrieval: French-English Dynamic Test Collection for Continuous Web Search Evaluationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 2; 2026: 2.

6 papers.

  1. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  2. 2025LegalSearchLM: Rethinking Legal Case Retrieval as Legal Elements Generationcited by 5
  3. 2026Can Large Language Models Keep Up? Benchmarking Online Adaptation to Continual Knowledge Streamscited by 5
  4. 2025MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corporacited by 4
  5. 2023Back to Basics: A Simple Recipe for Improving Out-of-Domain Retrieval in Dense Encoderscited by 1
  6. 2026Generative Universal Multimodal Retrieval with Dual-role Identifierscited by 0