# GenQREnsemble: Zero-Shot LLM Ensemble Prompting for Generative Query Reformulation

arXiv: https://arxiv.org/abs/2404.03746 · HTML: https://mlsanity.com/paper/2404.03746/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 35 references that are in the map.
- First posted on arXiv 4 Apr 2024.

## Builds on (most cited first, 23 of 23)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [Instruction Tuning with GPT-4](https://mlsanity.com/paper/2304.03277/) (2023)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [Chain-of-Verification Reduces Hallucination in Large Language Models](https://mlsanity.com/paper/2309.11495/) (2023)
- [Making Large Language Models Better Reasoners with Step-Aware Verifier](https://mlsanity.com/paper/2206.02336/) (2022)
- [Ask Me Anything: A simple strategy for prompting language models](https://mlsanity.com/paper/2210.02441/) (2022)
- [On Decoding Strategies for Neural Text Generators](https://mlsanity.com/paper/2203.15721/) (2022)
- [The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models](https://mlsanity.com/paper/2101.05667/) (2021)
- [NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation](https://mlsanity.com/paper/2112.02721/) (2021)
- [ConvGQR: Generative Query Reformulation for Conversational Search](https://mlsanity.com/paper/2305.15645/) (2023)
- [Query Expansion by Prompting Large Language Models](https://mlsanity.com/paper/2305.03653/) (2023)
- [Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback](https://mlsanity.com/paper/2108.13454/) (2021)
- [Pseudo Relevance Feedback with Deep Language Models and Dense Retrievers: Successes and Pitfalls](https://mlsanity.com/paper/2108.11044/) (2021)
- [When do Generative Query and Document Expansions Fail? A Comprehensive Study Across Methods, Retrievers, and Datasets](https://mlsanity.com/paper/2309.08541/) (2023)
- [Generative Query Reformulation for Effective Adhoc Search](https://mlsanity.com/paper/2308.00415/) (2023)
- [Diversity driven Query Rewriting in Search Advertising](https://mlsanity.com/paper/2106.03816/) (2021)

## Built on by (most cited first, 23 of 23)

- [A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challenges](https://mlsanity.com/paper/2508.05668/) (2025)
- [When Search Engine Services meet Large Language Models: Visions and Challenges](https://mlsanity.com/paper/2407.00128/) (2024)
- [Elaborative Subtopic Query Reformulation for Broad and Indirect Queries in Travel Destination Recommendation](https://mlsanity.com/paper/2410.01598/) (2024)
- [RRADistill: Distilling LLMs' Passage Ranking Ability for Long-Tail Queries Document Re-Ranking on a Search Engine](https://mlsanity.com/paper/2410.18097/) (2024)
- [Generative Query Reformulation Using Ensemble Prompting, Document Fusion, and Relevance Feedback](https://mlsanity.com/paper/2405.17658/) (2024)
- [Database-Augmented Query Representation for Information Retrieval](https://mlsanity.com/paper/2406.16013/) (2024)
- [Generative Product Recommendations for Implicit Superlative Queries](https://mlsanity.com/paper/2504.18748/) (2025)
- [QueryGym: A Toolkit for Reproducible LLM-Based Query Reformulation](https://mlsanity.com/paper/2511.15996/) (2025)
- [Broadening Access to Simulations for End-Users via Large Language Models: Challenges and Opportunities](https://mlsanity.com/paper/2409.15290/) (2024)
- [Beyond Contrastive Learning: Synthetic Data Enables List-wise Training with Multiple Levels of Relevance](https://mlsanity.com/paper/2503.23239/) (2025)
- [A Reproducibility Study of LLM-Based Query Reformulation](https://mlsanity.com/paper/2604.27421/) (2026)
- [GenCRF: Generative Clustering and Reformulation Framework for Enhanced Intent-Driven Information Retrieval](https://mlsanity.com/paper/2409.10909/) (2024)
- [A Survey of Query Optimization in Large Language Models](https://mlsanity.com/paper/2412.17558/) (2024)
- [UiS-IAI@LiveRAG: Retrieval-Augmented Information Nugget-Based Generation of Responses](https://mlsanity.com/paper/2506.22210/) (2025)
- [Hint-Augmented Re-ranking: Efficient Product Search using LLM-Based Query Decomposition](https://mlsanity.com/paper/2511.13994/) (2025)
- [ReFormeR: Learning and Applying Explicit Query Reformulation Patterns](https://mlsanity.com/paper/2604.01417/) (2026)
- [Can QPP Choose the Right Query Variant? Evaluating Query Variant Selection for RAG Pipelines](https://mlsanity.com/paper/2604.22661/) (2026)
- [ADORE: Iterative Query Expansion with Retrieval-Grounded Relevance Feedback](https://mlsanity.com/paper/2606.13905/) (2026)
- [A Systematic Study of Pseudo-Relevance Feedback with LLMs](https://mlsanity.com/paper/2603.11008/) (2026)
- [ReBOL: Retrieval via Bayesian Optimization with Batched LLM Relevance Observations and Query Reformulation](https://mlsanity.com/paper/2603.20513/) (2026)
- [Seeking Information with RAG-Assistants: Does Model Size Matter in Human-AI Collaborations?](https://mlsanity.com/paper/2605.00964/) (2026)
- [A Unified Structured Query Understanding Framework for Industrial Semantic Search](https://mlsanity.com/paper/2605.27441/) (2026)
- [Beyond Semantic Similarity: Performance and Costs of Agentic Retrieval for Complex Tasks](https://mlsanity.com/paper/2610.05750/) (2026)

## Abstract (arXiv)

Query Reformulation(QR) is a set of techniques used to transform a user's original search query to a text that better aligns with the user's intent and improves their search experience. Recently, zero-shot QR has been shown to be a promising approach due to its ability to exploit knowledge inherent in large language models. By taking inspiration from the success of ensemble prompting strategies which have benefited many tasks, we investigate if they can help improve query reformulation. In this context, we propose an ensemble based prompting technique, GenQREnsemble which leverages paraphrases of a zero-shot instruction to generate multiple sets of keywords ultimately improving retrieval performance. We further introduce its post-retrieval variant, GenQREnsembleRF to incorporate pseudo relevant feedback. On evaluations over four IR benchmarks, we find that GenQREnsemble generates better reformulations with relative nDCG@10 improvements up to 18% and MAP improvements upto 24% over the previous zero-shot state-of-art. On the MSMarco Passage Ranking task, GenQREnsembleRF shows relative gains of 5% MRR using pseudo-relevance feedback, and 9% nDCG@10 using relevant feedback documents.