mlsanityData to 14 Oct 2026

On the Robustness of Generative Retrieval Models: An Out-of-Distribution Perspective

arXiv 2306.12756 ↗First posted 22 Jun 2023cs.IR cs.AI cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Recently, we have witnessed generative retrieval increasingly gaining attention in the information retrieval (IR) field, which retrieves documents by directly generating their identifiers. So far, much effort has been devoted to developing effective generative retrieval models. There has been less attention paid to the robustness perspective. When a new retrieval paradigm enters into the real-world application, it is also critical to measure the out-of-distribution (OOD) generalization, i.e., how would generative retrieval models generalize to new distributions. To answer this question, firstly, we define OOD robustness from three perspectives in retrieval problems: 1) The query variations; 2) The unforeseen query types; and 3) The unforeseen tasks. Based on this taxonomy, we conduct empirical studies to analyze the OOD robustness of several representative generative retrieval models against dense retrieval models. The empirical results indicate that the OOD robustness of generative retrieval models requires enhancement. We hope studying the OOD robustness of generative retrieval models would be advantageous to the IR community.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 30 of its 37 references that are in the map. First posted on arXiv 22 Jun 2023.

Builds on

30 of 37 references.

References · 30

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  6. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  7. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  8. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  9. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
  10. 2020Autoregressive Entity Retrievalcited by 334
  11. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 294
  12. 2022Transformer Memory as a Differentiable Search Indexcited by 247
18 more
  1. 2017A Survey on Dialogue Systems: Recent Advances and New Frontierscited by 162
  2. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  3. 2021Optimizing Dense Retrieval Model Training with Hard Negativescited by 155
  4. 2016Counter-fitting Word Vectors to Linguistic Constraintscited by 140
  5. 2022Autoregressive Search Engines: Generating Substrings as Document Identifierscited by 113
  6. 2022A Neural Corpus Indexer for Document Retrievalcited by 111
  7. 2016Learning to Match Using Local and Distributed Representations of Text for Web Searchcited by 68
  8. 2021Semantic Models for the First-stage Retrieval: A Comprehensive Reviewcited by 57
  9. 2022GERE: Generative Evidence Retrieval for Fact Verificationcited by 38
  10. 2022CorpusBrain: Pre-train a Generative Retrieval Model for Knowledge-Intensive Language Taskscited by 36
  11. 2021Evaluating the Robustness of Retrieval Pipelines with Query Variation Generatorscited by 34
  12. 2020Embedding-based Zero-shot Retrieval through Query Generationcited by 24
  13. 2021Dealing with Typos for BERT-based Passage Retrieval and Rankingcited by 23
  14. 2022Analysing the Robustness of Dual Encoders for Dense Retrieval Against Misspellingscited by 23
  15. 2023Topic-oriented Adversarial Attacks against Black-box Neural Ranking Modelscited by 18
  16. 2021Are Neural Ranking Models Robust?cited by 16
  17. 2023A Unified Generative Retriever for Knowledge-Intensive Language Tasks via Prompt Learningcited by 16
  18. 2022A Contrastive Pre-training Approach to Learn Discriminative Autoencoder for Dense Retrievalcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 2; 2026: 1.

8 papers.

  1. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  2. 2023Black-box Adversarial Attacks against Dense Retrieval Models: A Multi-view Contrastive Learning Methodcited by 29
  3. 2024Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspectivecited by 15
  4. 2024Multi-granular Adversarial Attacks against Black-box Neural Ranking Modelscited by 14
  5. 2025Investigating the Robustness of Retrieval-Augmented Generation at the Query Levelcited by 11
  6. 2023Perturbation-Invariant Adversarial Training for Neural Ranking Models: Improving the Effectiveness-Robustness Trade-Offcited by 7
  7. 2025Improving Document Retrieval Coherence for Semantically Equivalent Queriescited by 0
  8. 2026STELLA: Self-Reflective Terminology-Aware Framework for Building an Aerospace Information Retrieval Benchmarkcited by 0