# Deep Reinforced Query Reformulation for Information Retrieval

arXiv: https://arxiv.org/abs/2007.07987 · HTML: https://mlsanity.com/paper/2007.07987/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2020.

## Built on by (most cited first, 10 of 10)

- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Context Aware Query Rewriting for Text Rankers using LLM](https://mlsanity.com/paper/2308.16753/) (2023)
- [Query Understanding in the Age of Large Language Models](https://mlsanity.com/paper/2306.16004/) (2023)
- [Deep Literature Survey Automation with an Iterative Workflow](https://mlsanity.com/paper/2510.21900/) (2025)
- [Improved IR-based Bug Localization with Intelligent Relevance Feedback](https://mlsanity.com/paper/2501.10542/) (2025)
- [High Quality Related Search Query Suggestions using Deep Reinforcement Learning](https://mlsanity.com/paper/2108.04452/) (2021)
- [Rethinking On-policy Optimization for Query Augmentation](https://mlsanity.com/paper/2510.17139/) (2025)
- [From RAG to QA-RAG: Integrating Generative AI for Pharmaceutical Regulatory Compliance Process](https://mlsanity.com/paper/2402.01717/) (2024)
- [Improving Document Retrieval Coherence for Semantically Equivalent Queries](https://mlsanity.com/paper/2508.07975/) (2025)
- [MVR-cache: Optimizing Semantic Caching via Multi-Vector Retrieval and Learned Prompt Segmentation](https://mlsanity.com/paper/2605.24914/) (2026)

## Abstract (arXiv)

Query reformulations have long been a key mechanism to alleviate the vocabulary-mismatch problem in information retrieval, for example by expanding the queries with related query terms or by generating paraphrases of the queries. In this work, we propose a deep reinforced query reformulation (DRQR) model to automatically generate new reformulations of the query. To encourage the model to generate queries which can achieve high performance when performing the retrieval task, we incorporate query performance prediction into our reward function. In addition, to evaluate the quality of the reformulated query in the context of information retrieval, we first train our DRQR model, then apply the retrieval ranking model on the obtained reformulated query. Experiments are conducted on the TREC 2020 Deep Learning track MSMARCO document ranking dataset. Our results show that our proposed model outperforms several query reformulation model baselines when performing retrieval task. In addition, improvements are also observed when combining with various retrieval models, such as query expansion and BERT.