# Large Language Models are Strong Zero-Shot Retriever

arXiv: https://arxiv.org/abs/2304.14233 · HTML: https://mlsanity.com/paper/2304.14233/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 47 references that are in the map.
- First posted on arXiv 27 Apr 2023.

## Builds on (most cited first, 30 of 40)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [Learning To Retrieve Prompts for In-Context Learning](https://mlsanity.com/paper/2112.08633/) (2021)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [Large Dual Encoders Are Generalizable Retrievers](https://mlsanity.com/paper/2112.07899/) (2021)
- [Generate rather than Retrieve: Large Language Models are Strong Context Generators](https://mlsanity.com/paper/2209.10063/) (2022)
- [Overview of the TREC 2019 deep learning track](https://mlsanity.com/paper/2003.07820/) (2020)
- [Promptagator: Few-shot Dense Retrieval From 8 Examples](https://mlsanity.com/paper/2209.11755/) (2022)
- [SGPT: GPT Sentence Embeddings for Semantic Search](https://mlsanity.com/paper/2202.08904/) (2022)
- [Overview of the TREC 2020 deep learning track](https://mlsanity.com/paper/2102.07662/) (2021)
- [Query2doc: Query Expansion with Large Language Models](https://mlsanity.com/paper/2303.07678/) (2023)
- [Rethinking with Retrieval: Faithful Large Language Model Inference](https://mlsanity.com/paper/2301.00303/) (2022)
- [Knowledge-Grounded Dialogue Generation with Pre-trained Language Models](https://mlsanity.com/paper/2010.08824/) (2020)
- [Task-aware Retrieval with Instructions](https://mlsanity.com/paper/2211.09260/) (2022)
- [SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrieval](https://mlsanity.com/paper/2207.02578/) (2022)

## Built on by (most cited first, 26 of 26)

- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [A Survey on Knowledge Distillation of Large Language Models](https://mlsanity.com/paper/2402.13116/) (2024)
- [LLatrieval: LLM-Verified Retrieval for Verifiable Generation](https://mlsanity.com/paper/2311.07838/) (2023)
- [Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrieval](https://mlsanity.com/paper/2408.01875/) (2024)
- [RaFe: Ranking Feedback Improves Query Rewriting for RAG](https://mlsanity.com/paper/2405.14431/) (2024)
- [UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systems](https://mlsanity.com/paper/2401.13256/) (2024)
- [Re-Reading Improves Reasoning in Large Language Models](https://mlsanity.com/paper/2309.06275/) (2023)
- [GAR-meets-RAG Paradigm for Zero-Shot Information Retrieval](https://mlsanity.com/paper/2310.20158/) (2023)
- [Agent4Ranking: Semantic Robust Ranking via Personalized Query Rewriting Using Multi-agent LLM](https://mlsanity.com/paper/2312.15450/) (2023)
- [Question-Based Retrieval using Atomic Units for Enterprise RAG](https://mlsanity.com/paper/2405.12363/) (2024)
- [AutoMIR: Effective Zero-Shot Medical Information Retrieval without Relevance Labels](https://mlsanity.com/paper/2410.20050/) (2024)
- [Disentangling Extraction and Reasoning in Multi-hop Spatial Reasoning](https://mlsanity.com/paper/2310.16731/) (2023)
- [W-RAG: Weakly Supervised Dense Retrieval in RAG for Open-domain Question Answering](https://mlsanity.com/paper/2408.08444/) (2024)
- [Meta Knowledge for Retrieval Augmented Large Language Models](https://mlsanity.com/paper/2408.09017/) (2024)
- [Bayesian Active Learning with Gaussian Processes Guided by LLM Relevance Scoring for Dense Passage Retrieval](https://mlsanity.com/paper/2604.17906/) (2026)
- [ICXML: An In-Context Learning Framework for Zero-Shot Extreme Multi-Label Classification](https://mlsanity.com/paper/2311.09649/) (2023)
- [Beyond Contrastive Learning: Synthetic Data Enables List-wise Training with Multiple Levels of Relevance](https://mlsanity.com/paper/2503.23239/) (2025)
- [Referral Augmentation for Zero-Shot Information Retrieval](https://mlsanity.com/paper/2305.15098/) (2023)
- [Beyond Single-Shot: Multi-step Tool Retrieval via Query Planning](https://mlsanity.com/paper/2601.07782/) (2026)
- [HEISIR: Hierarchical Expansion of Inverted Semantic Indexing for Training-free Retrieval of Conversational Data using LLMs](https://mlsanity.com/paper/2503.04141/) (2025)
- [LMAR: Language Model Augmented Retriever for Domain-specific Knowledge Indexing](https://mlsanity.com/paper/2508.05672/) (2025)
- [ConceptCarve: Dynamic Realization of Evidence](https://mlsanity.com/paper/2504.07228/) (2025)
- [A Tale of LLMs and Induced Small Proxies: Scalable Small Language Models for Knowledge Mining](https://mlsanity.com/paper/2510.01427/) (2025)
- [LegalMALR:Multi-Agent Query Understanding and LLM-Based Reranking for Chinese Statute Retrieval](https://mlsanity.com/paper/2601.17692/) (2026)
- [Extreme Meta-Classification for Large-Scale Zero-Shot Retrieval](https://mlsanity.com/paper/2606.25237/) (2026)
- [Tracing and Taming Hallucinated Terms via Subsequence-Association Filtering and Breadth-Aware Selection for LLM Query Expansion](https://mlsanity.com/paper/2610.13810/) (2026)

## Abstract (arXiv)

In this work, we propose a simple method that applies a large language model (LLM) to large-scale retrieval in zero-shot scenarios. Our method, the Language language model as Retriever (LameR), is built upon no other neural models but an LLM, while breaking brute-force combinations of retrievers with LLMs and lifting the performance of zero-shot retrieval to be very competitive on benchmark datasets. Essentially, we propose to augment a query with its potential answers by prompting LLMs with a composition of the query and the query's in-domain candidates. The candidates, regardless of correct or wrong, are obtained by a vanilla retrieval procedure on the target collection. As a part of the prompts, they are likely to help LLM generate more precise answers by pattern imitation or candidate summarization. Even if all the candidates are wrong, the prompts at least make LLM aware of in-collection patterns and genres. Moreover, due to the low performance of a self-supervised retriever, the LLM-based query augmentation becomes less effective as the retriever bottlenecks the whole pipeline. Therefore, we propose to leverage a non-parametric lexicon-based method (e.g., BM25) as the retrieval module to capture query-document overlap in a literal fashion. As such, LameR makes the retrieval procedure transparent to the LLM, thus circumventing the performance bottleneck.