# PromptReps: Prompting Large Language Models to Generate Dense and Sparse Representations for Zero-Shot Document Retrieval

arXiv: https://arxiv.org/abs/2404.18424 · HTML: https://mlsanity.com/paper/2404.18424/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2024.

## Built on by (most cited first, 30 of 30)

- [MM-Embed: Universal Multimodal Retrieval with Multimodal LLMs](https://mlsanity.com/paper/2411.02571/) (2024)
- [Making Text Embedders Few-Shot Learners](https://mlsanity.com/paper/2409.15700/) (2024)
- [LLMs are Also Effective Embedding Models: An In-depth Overview](https://mlsanity.com/paper/2412.12591/) (2024)
- [Large Language Models as Foundations for Next-Gen Dense Retrieval: A Comprehensive Empirical Assessment](https://mlsanity.com/paper/2408.12194/) (2024)
- [Following the Autoregressive Nature of LLM Embeddings via Compression and Alignment](https://mlsanity.com/paper/2502.11401/) (2025)
- [FLAME: Frozen Large Language Models Enable Data-Efficient Language-Image Pre-training](https://mlsanity.com/paper/2411.11927/) (2024)
- [Crafting Interpretable Embeddings by Asking LLMs Questions](https://mlsanity.com/paper/2405.16714/) (2024)
- [CSPLADE: Learned Sparse Retrieval with Causal Language Models](https://mlsanity.com/paper/2504.10816/) (2025)
- [An Investigation of Prompt Variations for Zero-shot LLM-based Rankers](https://mlsanity.com/paper/2406.14117/) (2024)
- [AGENT-CQ: Automatic Generation and Evaluation of Clarifying Questions for Conversational Search with LLMs](https://mlsanity.com/paper/2410.19692/) (2024)
- [Enhancing Lexicon-Based Text Embeddings with Large Language Models](https://mlsanity.com/paper/2501.09749/) (2025)
- [Distillation versus Contrastive Learning: How to Train Your Rerankers](https://mlsanity.com/paper/2507.08336/) (2025)
- [CSRv2: Unlocking Ultra-Sparse Embeddings](https://mlsanity.com/paper/2602.05735/) (2026)
- [SkillSight: Calibrating Generic Content Bias for Skill Retrieval](https://mlsanity.com/paper/2607.18785/) (2026)
- [Zero-Shot Dense Retrieval with Embeddings from Relevance Feedback](https://mlsanity.com/paper/2410.21242/) (2024)
- [On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Survey](https://mlsanity.com/paper/2507.20783/) (2025)
- [Attention Grounded Enhancement for Visual Document Retrieval](https://mlsanity.com/paper/2511.13415/) (2025)
- [ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents](https://mlsanity.com/paper/2608.19662/) (2026)
- [LLM-VPRF: Large Language Model Based Vector Pseudo Relevance Feedback](https://mlsanity.com/paper/2504.01448/) (2025)
- [Document-Level Tabular Numerical Cross-Checking: A Coarse-to-Fine Approach](https://mlsanity.com/paper/2506.13328/) (2025)
- [Exploring Reasoning-Infused Text Embedding with Large Language Models for Zero-Shot Dense Retrieval](https://mlsanity.com/paper/2509.00276/) (2025)
- [AttnCache: Accelerating Self-Attention Inference for LLM Prefill via Attention Cache](https://mlsanity.com/paper/2510.25979/) (2025)
- [LightRetriever: A LLM-based Text Retrieval Architecture with Extremely Faster Query Inference](https://mlsanity.com/paper/2505.12260/) (2025)
- [A Comparative Study of Specialized LLMs as Dense Retrievers](https://mlsanity.com/paper/2507.03958/) (2025)
- [Context-Adaptive Multi-Prompt Embedding with Large Language Models for Vision-Language Alignment](https://mlsanity.com/paper/2508.02762/) (2025)
- [DiffRetriever: Parallel Representative Tokens for Retrieval with Diffusion Language Models](https://mlsanity.com/paper/2605.07210/) (2026)
- [HyReC: Exploring Hybrid-based Retriever for Chinese](https://mlsanity.com/paper/2506.21913/) (2025)
- [Safe to Check, Unsafe to Use: Relinking at the Compression Boundary of LLM Agents](https://mlsanity.com/paper/2606.21732/) (2026)
- [PRecG: Legal Precedent Retrieval with Graph Neural Networks and Rhetorical Role Segmentation](https://mlsanity.com/paper/2607.09094/) (2026)
- [Effective Dense Retrieval using Only In-Context Examples](https://mlsanity.com/paper/2609.38099/) (2026)

## Abstract (arXiv)

Utilizing large language models (LLMs) for zero-shot document ranking is done in one of two ways: (1) prompt-based re-ranking methods, which require no further training but are only feasible for re-ranking a handful of candidate documents due to computational costs; and (2) unsupervised contrastive trained dense retrieval methods, which can retrieve relevant documents from the entire corpus but require a large amount of paired text data for contrastive training. In this paper, we propose PromptReps, which combines the advantages of both categories: no need for training and the ability to retrieve from the whole corpus. Our method only requires prompts to guide an LLM to generate query and document representations for effective document retrieval. Specifically, we prompt the LLMs to represent a given text using a single word, and then use the last token's hidden states and the corresponding logits associated with the prediction of the next token to construct a hybrid document retrieval system. The retrieval system harnesses both dense text embedding and sparse bag-of-words representations given by the LLM. Our experimental evaluation on the MSMARCO, TREC deep learning and BEIR zero-shot document retrieval datasets illustrates that this simple prompt-based LLM retrieval method can achieve a similar or higher retrieval effectiveness than state-of-the-art LLM embedding methods that are trained with large amounts of unsupervised data, especially when using a larger LLM.