# Rank1: Test-Time Compute for Reranking in Information Retrieval

arXiv: https://arxiv.org/abs/2502.18418 · HTML: https://mlsanity.com/paper/2502.18418/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 37 of its 51 references that are in the map.
- First posted on arXiv 25 Feb 2025.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 30 of 37)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [Multilingual E5 Text Embeddings: A Technical Report](https://mlsanity.com/paper/2402.05672/) (2024)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://mlsanity.com/paper/2304.09542/) (2023)
- [One Embedder, Any Task: Instruction-Finetuned Text Embeddings](https://mlsanity.com/paper/2212.09741/) (2022)
- [Overview of the TREC 2019 deep learning track](https://mlsanity.com/paper/2003.07820/) (2020)
- [Generative Representational Instruction Tuning](https://mlsanity.com/paper/2402.09906/) (2024)
- [Document Expansion by Query Prediction](https://mlsanity.com/paper/1904.08375/) (2019)
- [BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval](https://mlsanity.com/paper/2407.12883/) (2024)
- [RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!](https://mlsanity.com/paper/2312.02724/) (2023)
- [MMTEB: Massive Multilingual Text Embedding Benchmark](https://mlsanity.com/paper/2502.13595/) (2025)
- [Task-aware Retrieval with Instructions](https://mlsanity.com/paper/2211.09260/) (2022)
- [BM25S: Orders of magnitude faster lexical search via eager sparse scoring](https://mlsanity.com/paper/2407.03618/) (2024)
- [Gecko: Versatile Text Embeddings Distilled from Large Language Models](https://mlsanity.com/paper/2403.20327/) (2024)
- [RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models](https://mlsanity.com/paper/2309.15088/) (2023)
- [FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions](https://mlsanity.com/paper/2403.15246/) (2024)
- [Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models](https://mlsanity.com/paper/2409.11136/) (2024)
- [RAR-b: Reasoning as Retrieval Benchmark](https://mlsanity.com/paper/2404.06347/) (2024)
- [JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking](https://mlsanity.com/paper/2411.00142/) (2024)
- [INSTRUCTIR: A Benchmark for Instruction Following of Information Retrieval Models](https://mlsanity.com/paper/2402.14334/) (2024)
- [Is That Your Final Answer? Test-Time Scaling Improves Selective Question Answering](https://mlsanity.com/paper/2502.13962/) (2025)

## Built on by (most cited first, 30 of 70)

- [On the Theoretical Limitations of Embedding-Based Retrieval](https://mlsanity.com/paper/2508.21038/) (2025)
- [ReasonIR: Training Retrievers for Reasoning Tasks](https://mlsanity.com/paper/2504.20595/) (2025)
- [DIVER: A Multi-Stage Approach for Reasoning-intensive Information Retrieval](https://mlsanity.com/paper/2508.07995/) (2025)
- [RAG-RL: Advancing Retrieval-Augmented Generation via RL and Curriculum Learning](https://mlsanity.com/paper/2503.12759/) (2025)
- [Rank-K: Test-Time Reasoning for Listwise Reranking](https://mlsanity.com/paper/2505.14432/) (2025)
- [RaDeR: Reasoning-aware Dense Retrieval Models](https://mlsanity.com/paper/2505.18405/) (2025)
- [Table-R1: Inference-Time Scaling for Table Reasoning](https://mlsanity.com/paper/2505.23621/) (2025)
- [IFIR: A Comprehensive Benchmark for Evaluating Instruction-Following in Expert-Domain Information Retrieval](https://mlsanity.com/paper/2503.04644/) (2025)
- [ReasonEmbed: Enhanced Text Embeddings for Reasoning-Intensive Document Retrieval](https://mlsanity.com/paper/2510.08252/) (2025)
- [WikiVideo: Article Generation from Multiple Videos](https://mlsanity.com/paper/2504.00939/) (2025)
- [ThinkQE: Query Expansion via an Evolving Thinking Process](https://mlsanity.com/paper/2506.09260/) (2025)
- [MRMR: A Realistic and Expert-Level Multidisciplinary Benchmark for Reasoning-Intensive Multimodal Retrieval](https://mlsanity.com/paper/2510.09510/) (2025)
- [ProRank: Prompt Warmup via Reinforcement Learning for Small Language Models Reranking](https://mlsanity.com/paper/2506.03487/) (2025)
- [TongSearch-QR: Reinforced Query Reasoning for Retrieval](https://mlsanity.com/paper/2506.11603/) (2025)
- [Optimizing Generative Ranking Relevance via Reinforcement Learning in Xiaohongshu Search](https://mlsanity.com/paper/2512.00968/) (2025)
- [LeanSearch v2: Global Premise Retrieval for Lean 4 Theorem Proving](https://mlsanity.com/paper/2605.13137/) (2026)
- [AcuRank: Uncertainty-Aware Adaptive Computation for Listwise Reranking](https://mlsanity.com/paper/2505.18512/) (2025)
- [Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval](https://mlsanity.com/paper/2602.19961/) (2026)
- [Don't "Overthink" Passage Reranking: Is Reasoning Truly Necessary?](https://mlsanity.com/paper/2505.16886/) (2025)
- [Retro*: Optimizing LLMs for Reasoning-Intensive Document Retrieval](https://mlsanity.com/paper/2509.24869/) (2025)
- [MR2-Bench: Going Beyond Matching to Reasoning in Multimodal Retrieval](https://mlsanity.com/paper/2509.26378/) (2025)
- [Rethinking Reasoning in Document Ranking: Why Chain-of-Thought Falls Short](https://mlsanity.com/paper/2510.08985/) (2025)
- [SAGE: Benchmarking and Improving Retrieval for Deep Research Agents](https://mlsanity.com/paper/2602.05975/) (2026)
- [ERank: Fusing Supervised Fine-Tuning and Reinforcement Learning for Effective and Efficient Text Reranking](https://mlsanity.com/paper/2509.00520/) (2025)
- [Benchmarking Information Retrieval Models on Complex Retrieval Tasks](https://mlsanity.com/paper/2509.07253/) (2025)
- [TFRank: Think-Free Reasoning Enables Practical Pointwise LLM Ranking](https://mlsanity.com/paper/2508.09539/) (2025)
- [Beyond Sequential Reranking: Reranker-Guided Search Improves Reasoning Intensive Retrieval](https://mlsanity.com/paper/2509.07163/) (2025)
- [The Embedder's Dilemma: LLMs Are Better, but at What Cost?](https://mlsanity.com/paper/2608.12875/) (2026)
- [Beyond Contrastive Learning: Synthetic Data Enables List-wise Training with Multiple Levels of Relevance](https://mlsanity.com/paper/2503.23239/) (2025)
- [InsertRank: LLMs can reason over BM25 scores to Improve Listwise Reranking](https://mlsanity.com/paper/2506.14086/) (2025)

## Abstract (arXiv)

We introduce Rank1, the first reranking model trained to take advantage of test-time compute. Rank1 demonstrates the applicability within retrieval of using a reasoning language model (i.e. OpenAI's o1, Deepseek's R1, etc.) for distillation in order to rapidly improve the performance of a smaller model. We gather and open-source a dataset of more than 600,000 examples of R1 reasoning traces from queries and passages in MS MARCO. Models trained on this dataset show: (1) state-of-the-art performance on advanced reasoning and instruction following datasets; (2) work remarkably well out of distribution due to the ability to respond to user-input prompts; and (3) have explainable reasoning chains that can be given to users or RAG-based systems. Further, we demonstrate that quantized versions of these models retain strong performance while using less compute/memory. Overall, Rank1 shows that test-time compute allows for a fundamentally new type of explainable and performant reranker model for search.