# RLIRank: Learning to Rank with Reinforcement Learning for Dynamic Search

arXiv: https://arxiv.org/abs/2105.10124 · HTML: https://mlsanity.com/paper/2105.10124/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2021.

## Built on by (most cited first, 7 of 7)

- [FRAMM: Fair Ranking with Missing Modalities for Clinical Trial Site Selection](https://mlsanity.com/paper/2305.19407/) (2023)
- [Leveraging Large Language Models for Enhanced Product Descriptions in eCommerce](https://mlsanity.com/paper/2310.18357/) (2023)
- [MA4DIV: Multi-Agent Reinforcement Learning for Search Result Diversification](https://mlsanity.com/paper/2403.17421/) (2024)
- [Fairness in Image Search: A Study of Occupational Stereotyping in Image Retrieval and its Debiasing](https://mlsanity.com/paper/2305.03881/) (2023)
- [A Deep Reinforcement Learning Approach for Interactive Search with Sentence-level Feedback](https://mlsanity.com/paper/2310.03043/) (2023)
- [Enhanced E-Commerce Attribute Extraction: Innovating with Decorative Relation Correction and LLAMA 2.0-Based Annotation](https://mlsanity.com/paper/2312.06684/) (2023)
- [DenoiseRank: Learning to Rank by Diffusion Models](https://mlsanity.com/paper/2604.20852/) (2026)

## Abstract (arXiv)

To support complex search tasks, where the initial information requirements are complex or may change during the search, a search engine must adapt the information delivery as the user's information requirements evolve. To support this dynamic ranking paradigm effectively, search result ranking must incorporate both the user feedback received, and the information displayed so far. To address this problem, we introduce a novel reinforcement learning-based approach, RLIrank. We first build an adapted reinforcement learning framework to integrate the key components of the dynamic search. Then, we implement a new Learning to Rank (LTR) model for each iteration of the dynamic search, using a recurrent Long Short Term Memory neural network (LSTM), which estimates the gain for each next result, learning from each previously ranked document. To incorporate the user's feedback, we develop a word-embedding variation of the classic Rocchio Algorithm, to help guide the ranking towards the high-value documents. Those innovations enable RLIrank to outperform the previously reported methods from the TREC Dynamic Domain Tracks 2017 and exceed all the methods in 2016 TREC Dynamic Domain after multiple search iterations, advancing the state of the art for dynamic search.