# REARANK: Reasoning Re-ranking Agent via Reinforcement Learning

arXiv: https://arxiv.org/abs/2505.20046 · HTML: https://mlsanity.com/paper/2505.20046/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 30 of its 43 references that are in the map.
- First posted on arXiv 26 May 2025.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)
- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 30 of 30)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [LIMO: Less is More for Reasoning](https://mlsanity.com/paper/2502.03387/) (2025)
- [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://mlsanity.com/paper/2405.17428/) (2024)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://mlsanity.com/paper/2404.05961/) (2024)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://mlsanity.com/paper/2304.09542/) (2023)
- [Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting](https://mlsanity.com/paper/2306.17563/) (2023)
- [Overview of the TREC 2019 deep learning track](https://mlsanity.com/paper/2003.07820/) (2020)
- [Improving Passage Retrieval with Zero-Shot Question Generation](https://mlsanity.com/paper/2204.07496/) (2022)
- [BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval](https://mlsanity.com/paper/2407.12883/) (2024)
- [RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!](https://mlsanity.com/paper/2312.02724/) (2023)
- [Holistic Evaluation of Language Models](https://mlsanity.com/paper/2211.09110/) (2022)
- [DeepSeek-R1 Thoughtology: Let's think about LLM Reasoning](https://mlsanity.com/paper/2504.07128/) (2025)
- [Zero-Shot Listwise Document Reranking with a Large Language Model](https://mlsanity.com/paper/2305.02156/) (2023)
- [RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models](https://mlsanity.com/paper/2309.15088/) (2023)
- [Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learning](https://mlsanity.com/paper/2503.06034/) (2025)
- [Assessing and Learning Alignment of Unimodal Vision and Language Models](https://mlsanity.com/paper/2412.04616/) (2024)
- [Exploring the Best Practices of Query Expansion with Large Language Models](https://mlsanity.com/paper/2401.06311/) (2024)
- [MoqaGPT : Zero-Shot Multi-modal Open-domain Question Answering with Large Language Model](https://mlsanity.com/paper/2310.13265/) (2023)

## Built on by (most cited first, 24 of 24)

- [DIVER: A Multi-Stage Approach for Reasoning-intensive Information Retrieval](https://mlsanity.com/paper/2508.07995/) (2025)
- [It Takes Two: Your GRPO Is Secretly DPO](https://mlsanity.com/paper/2510.00977/) (2025)
- [Mano Technical Report](https://mlsanity.com/paper/2509.17336/) (2025)
- [Optimizing Generative Ranking Relevance via Reinforcement Learning in Xiaohongshu Search](https://mlsanity.com/paper/2512.00968/) (2025)
- [GR2: Generative Reasoning Re-ranker](https://mlsanity.com/paper/2602.07774/) (2026)
- [Rethinking Reasoning in Document Ranking: Why Chain-of-Thought Falls Short](https://mlsanity.com/paper/2510.08985/) (2025)
- [Recursive Self-Evolving Agents via Held-Out Selection](https://mlsanity.com/paper/2606.28374/) (2026)
- [RoboRouter: Training-Free Policy Routing for Robotic Manipulation](https://mlsanity.com/paper/2603.07892/) (2026)
- [SciRerankBench: Benchmarking Rerankers Towards Scientific Retrieval-Augmented Generated LLMs](https://mlsanity.com/paper/2508.08742/) (2025)
- [TFRank: Think-Free Reasoning Enables Practical Pointwise LLM Ranking](https://mlsanity.com/paper/2508.09539/) (2025)
- [MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoning](https://mlsanity.com/paper/2603.03379/) (2026)
- [Beyond Relevance-Centric Retrieval: Rubric-Oriented Document Set Selection and Ranking](https://mlsanity.com/paper/2607.19747/) (2026)
- [CogniSQL-R1-Zero: Lightweight Reinforced Reasoning for Efficient SQL Generation](https://mlsanity.com/paper/2507.06013/) (2025)
- [LoRALib: A Standardized Benchmark for Evaluating LoRA-MoE Methods](https://mlsanity.com/paper/2509.18137/) (2025)
- [ToM: Leveraging Tree-oriented MapReduce for Long-Context Reasoning in Large Language Models](https://mlsanity.com/paper/2511.00489/) (2025)
- [The Past Is Not Past: Memory-Enhanced Dynamic Reward Shaping](https://mlsanity.com/paper/2604.11297/) (2026)
- [AdaTutoRank: Learning to Rerank Document Sets via Adaptive Tutoring Optimization for RAG and Deep Research](https://mlsanity.com/paper/2609.32472/) (2026)
- [DeepCodeSeek: Real-Time API Retrieval for Context-Aware Code Generation](https://mlsanity.com/paper/2509.25716/) (2025)
- [LIR3AG: A Lightweight Rerank Reasoning Strategy Framework for Retrieval-Augmented Generation](https://mlsanity.com/paper/2512.18329/) (2025)
- [GR2 Technical Report](https://mlsanity.com/paper/2606.31984/) (2026)
- [Diffusion-GR2: Diffusion Generative Reasoning Re-ranker](https://mlsanity.com/paper/2607.01170/) (2026)
- [HiFloat4 Format for End-To-End Reinforcement Learning Post-Training of Large Language Models](https://mlsanity.com/paper/2607.26515/) (2026)
- [Better Nearest Neighbor Graph Indices via (Efficient) LLM-Guided Pruning](https://mlsanity.com/paper/2609.36359/) (2026)
- [Component-Aware Feedback for Self-Evolving Programs](https://mlsanity.com/paper/2609.38639/) (2026)

## Abstract (arXiv)

We present REARANK, a large language model (LLM)-based listwise reasoning reranking agent. REARANK explicitly reasons before reranking, significantly improving both performance and interpretability. Leveraging reinforcement learning and data augmentation, REARANK achieves substantial improvements over baseline models across popular information retrieval benchmarks, notably requiring only 179 annotated samples. Built on top of Qwen2.5-7B, our REARANK-7B demonstrates performance comparable to GPT-4 on both in-domain and out-of-domain benchmarks and even surpasses GPT-4 on reasoning-intensive BRIGHT benchmarks. These results underscore the effectiveness of our approach and highlight how reinforcement learning can enhance LLM reasoning capabilities in reranking.