# Context Aware Query Rewriting for Text Rankers using LLM

arXiv: https://arxiv.org/abs/2308.16753 · HTML: https://mlsanity.com/paper/2308.16753/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 59 references that are in the map.
- First posted on arXiv 31 Aug 2023.

## Builds on (most cited first, 30 of 37)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [Larger language models do in-context learning differently](https://mlsanity.com/paper/2303.03846/) (2023)
- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA](https://mlsanity.com/paper/2109.05014/) (2021)
- [Generate rather than Retrieve: Large Language Models are Strong Context Generators](https://mlsanity.com/paper/2209.10063/) (2022)
- [Prompting GPT-3 To Be Reliable](https://mlsanity.com/paper/2210.09150/) (2022)
- [Transformer Memory as a Differentiable Search Index](https://mlsanity.com/paper/2202.06991/) (2022)
- [Promptagator: Few-shot Dense Retrieval From 8 Examples](https://mlsanity.com/paper/2209.11755/) (2022)
- [Document Expansion by Query Prediction](https://mlsanity.com/paper/1904.08375/) (2019)
- [Query Rewriting for Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2305.14283/) (2023)
- [Query2doc: Query Expansion with Large Language Models](https://mlsanity.com/paper/2303.07678/) (2023)
- [Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information](https://mlsanity.com/paper/1805.04655/) (2018)
- [Deeper Text Understanding for IR with Contextual Neural Language Modeling](https://mlsanity.com/paper/1905.09217/) (2019)
- [Autoregressive Search Engines: Generating Substrings as Document Identifiers](https://mlsanity.com/paper/2204.10628/) (2022)
- [Task-aware Retrieval with Instructions](https://mlsanity.com/paper/2211.09260/) (2022)
- [CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code](https://mlsanity.com/paper/2302.05527/) (2023)
- [InPars: Data Augmentation for Information Retrieval using Large Language Models](https://mlsanity.com/paper/2202.05144/) (2022)
- [Query Expansion by Prompting Large Language Models](https://mlsanity.com/paper/2305.03653/) (2023)
- [Generative Query Reformulation for Effective Adhoc Search](https://mlsanity.com/paper/2308.00415/) (2023)
- [Doc2Query--: When Less is More](https://mlsanity.com/paper/2301.03266/) (2023)
- [Rethinking Search: Making Domain Experts out of Dilettantes](https://mlsanity.com/paper/2105.02274/) (2021)

## Built on by (most cited first, 14 of 14)

- [System 2 Attention (is something you might need too)](https://mlsanity.com/paper/2311.11829/) (2023)
- [Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrieval](https://mlsanity.com/paper/2408.01875/) (2024)
- [On the Multi-turn Instruction Following for Conversational Web Agents](https://mlsanity.com/paper/2402.15057/) (2024)
- [HM-RAG: Hierarchical Multi-Agent Multimodal Retrieval Augmented Generation](https://mlsanity.com/paper/2504.12330/) (2025)
- [EVOR: Evolving Retrieval for Code Generation](https://mlsanity.com/paper/2402.12317/) (2024)
- [SPAR: Scholar Paper Retrieval with LLM-based Agents for Enhanced Academic Search](https://mlsanity.com/paper/2507.15245/) (2025)
- [Disambiguation in Conversational Question Answering in the Era of LLMs and Agents: A Survey](https://mlsanity.com/paper/2505.12543/) (2025)
- [A Reality check of the benefits of LLM in business](https://mlsanity.com/paper/2406.10249/) (2024)
- [Broadening Access to Simulations for End-Users via Large Language Models: Challenges and Opportunities](https://mlsanity.com/paper/2409.15290/) (2024)
- [Beyond Contrastive Learning: Synthetic Data Enables List-wise Training with Multiple Levels of Relevance](https://mlsanity.com/paper/2503.23239/) (2025)
- [KaPQA: Knowledge-Augmented Product Question-Answering](https://mlsanity.com/paper/2407.16073/) (2024)
- [Exploring Rewriting Approaches for Different Conversational Tasks](https://mlsanity.com/paper/2502.18860/) (2025)
- [PaperRegister: Boosting Flexible-grained Paper Search via Hierarchical Register Indexing](https://mlsanity.com/paper/2508.11116/) (2025)
- [Understanding the User: An Intent-Based Ranking Dataset](https://mlsanity.com/paper/2408.17103/) (2024)

## Abstract (arXiv)

Query rewriting refers to an established family of approaches that are applied to underspecified and ambiguous queries to overcome the vocabulary mismatch problem in document ranking. Queries are typically rewritten during query processing time for better query modelling for the downstream ranker. With the advent of large-language models (LLMs), there have been initial investigations into using generative approaches to generate pseudo documents to tackle this inherent vocabulary gap. In this work, we analyze the utility of LLMs for improved query rewriting for text ranking tasks. We find that there are two inherent limitations of using LLMs as query re-writers -- concept drift when using only queries as prompts and large inference costs during query processing. We adopt a simple, yet surprisingly effective, approach called context aware query rewriting (CAR) to leverage the benefits of LLMs for query understanding. Firstly, we rewrite ambiguous training queries by context-aware prompting of LLMs, where we use only relevant documents as context.Unlike existing approaches, we use LLM-based query rewriting only during the training phase. Eventually, a ranker is fine-tuned on the rewritten queries instead of the original queries during training. In our extensive experiments, we find that fine-tuning a ranker using re-written queries offers a significant improvement of up to 33% on the passage ranking task and up to 28% on the document ranking task when compared to the baseline performance of using original queries.