# A Neural Corpus Indexer for Document Retrieval

arXiv: https://arxiv.org/abs/2206.02743 · HTML: https://mlsanity.com/paper/2206.02743/

## Key facts

- Cited by 111 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2022.

## Built on by (most cited first, 30 of 111)

- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting](https://mlsanity.com/paper/2306.17563/) (2023)
- [Recommender Systems with Generative Retrieval](https://mlsanity.com/paper/2305.05065/) (2023)
- [Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models](https://mlsanity.com/paper/2212.08037/) (2022)
- [From Matching to Generation: A Survey on Generative Information Retrieval](https://mlsanity.com/paper/2404.14851/) (2024)
- [ToolGen: Unified Tool Retrieval and Calling via Generation](https://mlsanity.com/paper/2410.03439/) (2024)
- [Recitation-Augmented Language Models](https://mlsanity.com/paper/2210.01296/) (2022)
- [DeTeCtive: Detecting AI-generated Text via Multi-Level Contrastive Learning](https://mlsanity.com/paper/2410.20964/) (2024)
- [Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation](https://mlsanity.com/paper/2206.10128/) (2022)
- [How Does Generative Retrieval Scale to Millions of Passages?](https://mlsanity.com/paper/2305.11841/) (2023)
- [Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Models](https://mlsanity.com/paper/2305.09955/) (2023)
- [Learning to Rank in Generative Retrieval](https://mlsanity.com/paper/2306.15222/) (2023)
- [Source-Aware Training Enables Knowledge Attribution in Language Models](https://mlsanity.com/paper/2404.01019/) (2024)
- [Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications](https://mlsanity.com/paper/2311.05876/) (2023)
- [Large Language Models are Built-in Autoregressive Search Engines](https://mlsanity.com/paper/2305.09612/) (2023)
- [C-RAG: Certified Generation Risks for Retrieval-Augmented Language Models](https://mlsanity.com/paper/2402.03181/) (2024)
- [Generative Multi-Modal Knowledge Retrieval with Large Language Models](https://mlsanity.com/paper/2401.08206/) (2024)
- [Language Models As Semantic Indexers](https://mlsanity.com/paper/2310.07815/) (2023)
- [RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generation](https://mlsanity.com/paper/2412.11919/) (2024)
- [Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding](https://mlsanity.com/paper/2404.14600/) (2024)
- [IRGen: Generative Modeling for Image Retrieval](https://mlsanity.com/paper/2303.10126/) (2023)
- [Unifying Generative and Dense Retrieval for Sequential Recommendation](https://mlsanity.com/paper/2411.18814/) (2024)
- [GLEN: Generative Retrieval via Lexical Index Learning](https://mlsanity.com/paper/2311.03057/) (2023)
- [Multiview Identifiers Enhanced Generative Retrieval](https://mlsanity.com/paper/2305.16675/) (2023)
- [Constructing Tree-based Index for Efficient and Effective Dense Retrieval](https://mlsanity.com/paper/2304.11943/) (2023)
- [UniGen: A Unified Generative Framework for Retrieval and Question Answering with Large Language Models](https://mlsanity.com/paper/2312.11036/) (2023)
- [SSRL: Self-Search Reinforcement Learning](https://mlsanity.com/paper/2508.10874/) (2025)
- [In-Context Exemplars as Clues to Retrieving from Large Associative Memory](https://mlsanity.com/paper/2311.03498/) (2023)
- [IncDSI: Incrementally Updatable Document Retrieval](https://mlsanity.com/paper/2307.10323/) (2023)

## Abstract (arXiv)

Current state-of-the-art document retrieval solutions mainly follow an index-retrieve paradigm, where the index is hard to be directly optimized for the final retrieval target. In this paper, we aim to show that an end-to-end deep neural network unifying training and indexing stages can significantly improve the recall performance of traditional methods. To this end, we propose Neural Corpus Indexer (NCI), a sequence-to-sequence network that generates relevant document identifiers directly for a designated query. To optimize the recall performance of NCI, we invent a prefix-aware weight-adaptive decoder architecture, and leverage tailored techniques including query generation, semantic document identifiers, and consistency-based regularization. Empirical studies demonstrated the superiority of NCI on two commonly used academic benchmarks, achieving +21.4% and +16.8% relative enhancement for Recall@1 on NQ320k dataset and R-Precision on TriviaQA dataset, respectively, compared to the best baseline method.