# On the Robustness of Generative Retrieval Models: An Out-of-Distribution Perspective

arXiv: https://arxiv.org/abs/2306.12756 · HTML: https://mlsanity.com/paper/2306.12756/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 37 references that are in the map.
- First posted on arXiv 22 Jun 2023.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [Pretrained Transformers Improve Out-of-Distribution Robustness](https://mlsanity.com/paper/2004.06100/) (2020)
- [Transformer Memory as a Differentiable Search Index](https://mlsanity.com/paper/2202.06991/) (2022)
- [A Survey on Dialogue Systems: Recent Advances and New Frontiers](https://mlsanity.com/paper/1711.01731/) (2017)
- [Condenser: a Pre-training Architecture for Dense Retrieval](https://mlsanity.com/paper/2104.08253/) (2021)
- [Optimizing Dense Retrieval Model Training with Hard Negatives](https://mlsanity.com/paper/2104.08051/) (2021)
- [Counter-fitting Word Vectors to Linguistic Constraints](https://mlsanity.com/paper/1603.00892/) (2016)
- [Autoregressive Search Engines: Generating Substrings as Document Identifiers](https://mlsanity.com/paper/2204.10628/) (2022)
- [A Neural Corpus Indexer for Document Retrieval](https://mlsanity.com/paper/2206.02743/) (2022)
- [Learning to Match Using Local and Distributed Representations of Text for Web Search](https://mlsanity.com/paper/1610.08136/) (2016)
- [Semantic Models for the First-stage Retrieval: A Comprehensive Review](https://mlsanity.com/paper/2103.04831/) (2021)
- [GERE: Generative Evidence Retrieval for Fact Verification](https://mlsanity.com/paper/2204.05511/) (2022)
- [CorpusBrain: Pre-train a Generative Retrieval Model for Knowledge-Intensive Language Tasks](https://mlsanity.com/paper/2208.07652/) (2022)
- [Evaluating the Robustness of Retrieval Pipelines with Query Variation Generators](https://mlsanity.com/paper/2111.13057/) (2021)
- [Embedding-based Zero-shot Retrieval through Query Generation](https://mlsanity.com/paper/2009.10270/) (2020)
- [Dealing with Typos for BERT-based Passage Retrieval and Ranking](https://mlsanity.com/paper/2108.12139/) (2021)
- [Analysing the Robustness of Dual Encoders for Dense Retrieval Against Misspellings](https://mlsanity.com/paper/2205.02303/) (2022)
- [Topic-oriented Adversarial Attacks against Black-box Neural Ranking Models](https://mlsanity.com/paper/2304.14867/) (2023)
- [Are Neural Ranking Models Robust?](https://mlsanity.com/paper/2108.05018/) (2021)
- [A Unified Generative Retriever for Knowledge-Intensive Language Tasks via Prompt Learning](https://mlsanity.com/paper/2304.14856/) (2023)
- [A Contrastive Pre-training Approach to Learn Discriminative Autoencoder for Dense Retrieval](https://mlsanity.com/paper/2208.09846/) (2022)

## Built on by (most cited first, 8 of 8)

- [From Matching to Generation: A Survey on Generative Information Retrieval](https://mlsanity.com/paper/2404.14851/) (2024)
- [Black-box Adversarial Attacks against Dense Retrieval Models: A Multi-view Contrastive Learning Method](https://mlsanity.com/paper/2308.09861/) (2023)
- [Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2407.06992/) (2024)
- [Multi-granular Adversarial Attacks against Black-box Neural Ranking Models](https://mlsanity.com/paper/2404.01574/) (2024)
- [Investigating the Robustness of Retrieval-Augmented Generation at the Query Level](https://mlsanity.com/paper/2507.06956/) (2025)
- [Perturbation-Invariant Adversarial Training for Neural Ranking Models: Improving the Effectiveness-Robustness Trade-Off](https://mlsanity.com/paper/2312.10329/) (2023)
- [Improving Document Retrieval Coherence for Semantically Equivalent Queries](https://mlsanity.com/paper/2508.07975/) (2025)
- [STELLA: Self-Reflective Terminology-Aware Framework for Building an Aerospace Information Retrieval Benchmark](https://mlsanity.com/paper/2601.03496/) (2026)

## Abstract (arXiv)

Recently, we have witnessed generative retrieval increasingly gaining attention in the information retrieval (IR) field, which retrieves documents by directly generating their identifiers. So far, much effort has been devoted to developing effective generative retrieval models. There has been less attention paid to the robustness perspective. When a new retrieval paradigm enters into the real-world application, it is also critical to measure the out-of-distribution (OOD) generalization, i.e., how would generative retrieval models generalize to new distributions. To answer this question, firstly, we define OOD robustness from three perspectives in retrieval problems: 1) The query variations; 2) The unforeseen query types; and 3) The unforeseen tasks. Based on this taxonomy, we conduct empirical studies to analyze the OOD robustness of several representative generative retrieval models against dense retrieval models. The empirical results indicate that the OOD robustness of generative retrieval models requires enhancement. We hope studying the OOD robustness of generative retrieval models would be advantageous to the IR community.