mlsanityData to 14 Oct 2026

Nonparametric Decoding for Generative Retrieval

arXiv 2210.02068 ↗First posted 5 Oct 2022cs.IR cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

The generative retrieval model depends solely on the information encoded in its model parameters without external memory, its information capacity is limited and fixed. To overcome the limitation, we propose Nonparametric Decoding (Np Decoding) which can be applied to existing generative retrieval models. Np Decoding uses nonparametric contextualized vocab embeddings (external memory) rather than vanilla vocab embeddings as decoder vocab embeddings. By leveraging the contextualized vocab embeddings, the generative retrieval model is able to utilize both the parametric and nonparametric space. Evaluation over 9 datasets (8 single-hop and 1 multi-hop) in the document retrieval task shows that applying Np Decoding to generative retrieval models significantly improves the performance. We also show that Np Decoding is data- and parameter-efficient, and shows high performance in the zero-shot setting.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 29 of its 52 references that are in the map. First posted on arXiv 5 Oct 2022.

Builds on

29 of 52 references.

References · 29

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  6. 2017Billion-scale similarity search with GPUscited by 2,497
  7. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  8. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  9. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  10. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  11. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  12. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
17 more
  1. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  2. 2019ELI5: Long Form Question Answeringcited by 575
  3. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
  4. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  5. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
  6. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
  7. 2020Autoregressive Entity Retrievalcited by 334
  8. 2022Transformer Memory as a Differentiable Search Indexcited by 247
  9. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 214
  10. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  11. 2020Answering Complex Open-Domain Questions with Multi-Hop Dense Retrievalcited by 162
  12. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  13. 2022Autoregressive Search Engines: Generating Substrings as Document Identifierscited by 113
  14. 2022Training Language Models with Memory Augmentationcited by 113
  15. 2022A Neural Corpus Indexer for Document Retrievalcited by 111
  16. 2021Multi-task Retrieval for Knowledge-Intensive Taskscited by 44
  17. 2022Generative Multi-hop Retrievalcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 1; 2026: 2.

6 papers.

  1. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  2. 2024Unifying Generative and Dense Retrieval for Sequential Recommendationcited by 18
  3. 2025Lightweight and Direct Document Relevance Optimization for Generative Information Retrievalcited by 9
  4. 2023Back to Basics: A Simple Recipe for Improving Out-of-Domain Retrieval in Dense Encoderscited by 1
  5. 2026ICICLE: Expanding Retrieval with In-Context Documentscited by 0
  6. 2026Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Modelscited by 0