mlsanityData to 7 Oct 2026

Rank1: Test-Time Compute for Reranking in Information Retrieval

arXiv 2502.18418 ↗First posted 25 Feb 2025cs.IR cs.CL cs.LGcited by 68

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

We introduce Rank1, the first reranking model trained to take advantage of test-time compute. Rank1 demonstrates the applicability within retrieval of using a reasoning language model (i.e. OpenAI's o1, Deepseek's R1, etc.) for distillation in order to rapidly improve the performance of a smaller model. We gather and open-source a dataset of more than 600,000 examples of R1 reasoning traces from queries and passages in MS MARCO. Models trained on this dataset show: (1) state-of-the-art performance on advanced reasoning and instruction following datasets; (2) work remarkably well out of distribution due to the ability to respond to user-input prompts; and (3) have explainable reasoning chains that can be given to users or RAG-based systems. Further, we demonstrate that quantized versions of these models retain strong performance while using less compute/memory. Overall, Rank1 shows that test-time compute allows for a fundamentally new type of explainable and performant reranker model for search.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 37 of its 51 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

37 of 51 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking31 Oct
  2. 2024ReasoningRank: Teaching Student Models to Rank through Reasoning-Based Knowledge Distillation7 Oct
  3. 2024RAR-b: Reasoning as Retrieval Benchmark9 Apr

Other references · 34

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  4. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  5. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  6. 2025s1: Simple test-time scalingcited by 1,311
  7. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  8. 2024OpenAI o1 System Cardcited by 1,220
  9. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  10. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  11. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
  12. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 357
22 more
  1. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
  2. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  3. 2020Overview of the TREC 2019 deep learning trackcited by 282
  4. 2024Generative Representational Instruction Tuningcited by 227
  5. 2019Document Expansion by Query Predictioncited by 225
  6. 2024BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrievalcited by 153
  7. 2023RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!cited by 151
  8. 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 120
  9. 2022Task-aware Retrieval with Instructionscited by 107
  10. 2024BM25S: Orders of magnitude faster lexical search via eager sparse scoringcited by 87
  11. 2024Gecko: Versatile Text Embeddings Distilled from Large Language Modelscited by 81
  12. 2023RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Modelscited by 69
  13. 2024FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructionscited by 67
  14. 2024Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Modelscited by 39
  15. 2024INSTRUCTIR: A Benchmark for Instruction Following of Information Retrieval Modelscited by 32
  16. 2025Is That Your Final Answer? Test-Time Scaling Improves Selective Question Answeringcited by 17
  17. 2022NELLIE: A Neuro-Symbolic Inference Engine for Grounded, Compositional, and Explainable Reasoningcited by 16
  18. 2024Enhancing Systematic Decompositional Natural Language Inference Using Informal Logiccited by 12
  19. 2024Beyond Relevance: Evaluate and Improve Retrievers on Perspective Awarenesscited by 12
  20. 2023NeuralMind-UNICAMP at 2022 TREC NeuCLIR: Large Boring Rerankers for Cross-lingual Retrievalcited by 7
  21. 2025Reproducing NevIR: Negation in Neural Information Retrievalcited by 7
  22. 2022Can Old TREC Collections Reliably Evaluate Modern Neural Retrieval Models?cited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 35; 2026: 33.

68 papers.

Sort:

By area:

  1. 2025On the Theoretical Limitations of Embedding-Based Retrievalcited by 98
  2. 2025ReasonIR: Training Retrievers for Reasoning Taskscited by 76
  3. 2025DIVER: A Multi-Stage Approach for Reasoning-intensive Information Retrievalcited by 35
  4. 2025RAG-RL: Advancing Retrieval-Augmented Generation via RL and Curriculum Learningcited by 30
  5. 2025Rank-K: Test-Time Reasoning for Listwise Rerankingcited by 28
  6. 2025RaDeR: Reasoning-aware Dense Retrieval Modelscited by 27
  7. 2025Table-R1: Inference-Time Scaling for Table Reasoningcited by 23
  8. 2025IFIR: A Comprehensive Benchmark for Evaluating Instruction-Following in Expert-Domain Information Retrievalcited by 16
  9. 2025ReasonEmbed: Enhanced Text Embeddings for Reasoning-Intensive Document Retrievalcited by 15
  10. 2025WikiVideo: Article Generation from Multiple Videoscited by 13
  11. 2025MRMR: A Realistic and Expert-Level Multidisciplinary Benchmark for Reasoning-Intensive Multimodal Retrievalcited by 12
  12. 2025ProRank: Prompt Warmup via Reinforcement Learning for Small Language Models Rerankingcited by 11
38 more
  1. 2025ThinkQE: Query Expansion via an Evolving Thinking Processcited by 11
  2. 2025TongSearch-QR: Reinforced Query Reasoning for Retrievalcited by 11
  3. 2025Optimizing Generative Ranking Relevance via Reinforcement Learning in Xiaohongshu Searchcited by 11
  4. 2025AcuRank: Uncertainty-Aware Adaptive Computation for Listwise Rerankingcited by 8
  5. 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
  6. 2025Don't "Overthink" Passage Reranking: Is Reasoning Truly Necessary?cited by 7
  7. 2025Retro*: Optimizing LLMs for Reasoning-Intensive Document Retrievalcited by 7
  8. 2025MR2-Bench: Going Beyond Matching to Reasoning in Multimodal Retrievalcited by 7
  9. 2025Rethinking Reasoning in Document Ranking: Why Chain-of-Thought Falls Shortcited by 7
  10. 2026SAGE: Benchmarking and Improving Retrieval for Deep Research Agentscited by 7
  11. 2026LeanSearch v2: Global Premise Retrieval for Lean 4 Theorem Provingcited by 7
  12. 2025ERank: Fusing Supervised Fine-Tuning and Reinforcement Learning for Effective and Efficient Text Rerankingcited by 6
  13. 2025Benchmarking Information Retrieval Models on Complex Retrieval Taskscited by 6
  14. 2025TFRank: Think-Free Reasoning Enables Practical Pointwise LLM Rankingcited by 5
  15. 2025Beyond Sequential Reranking: Reranker-Guided Search Improves Reasoning Intensive Retrievalcited by 5
  16. 2025Beyond Contrastive Learning: Synthetic Data Enables List-wise Training with Multiple Levels of Relevancecited by 4
  17. 2025InsertRank: LLMs can reason over BM25 scores to Improve Listwise Rerankingcited by 4
  18. 2025Reason to Retrieve: Enhancing Query Understanding through Decomposition and Interpretationcited by 4
  19. 2026Taxonomy of the Retrieval System Framework: Pitfalls and Paradigmscited by 4
  20. 2026Learning from Emptiness: De-biasing Listwise Rerankers with Content-Agnostic Probability Calibrationcited by 4
  21. 2026The Embedder's Dilemma: LLMs Are Better, but at What Cost?cited by 4
  22. 2025Test-time Corpus Feedback: From Retrieval to RAGcited by 3
  23. 2025E2Rank: Unifying Text Embedding and Listwise Reranking for Effective and Efficient Searchcited by 3
  24. 2025LimRank: Less is More for Reasoning-Intensive Information Rerankingcited by 3
  25. 2025GroupRank: A Groupwise Paradigm for Effective and Efficient Passage Reranking with LLMscited by 3
  26. 2025The Evolution of Reranking Models in Information Retrieval: From Heuristic Methods to Large Language Modelscited by 3
  27. 2026OptiSet: Unified Optimizing Set Selection and Ranking for Retrieval-Augmented Generationcited by 3
  28. 2026Enough is as good as a feast: A Comprehensive Analysis of How Reinforcement Learning Mitigates Task Conflicts in LLMscited by 3
  29. 2025From Token to Action: State Machine Reasoning to Mitigate Overthinking in Information Retrievalcited by 2
  30. 2026IRPAPERS: A Visual Document Benchmark for Scientific Retrieval and Question Answeringcited by 2
  31. 2026Does Reasoning Make Search More Fair? Comparing Fairness in Reasoning and Non-Reasoning Rerankerscited by 2
  32. 2026CoverageBench: Evaluating Information Coverage across Tasks and Domainscited by 2
  33. 2026Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systemscited by 2
  34. 2025Test-Time Scaling Strategies for Generative Retrieval in Multimodal Conversational Recommendationscited by 1
  35. 2026RoutIR: Fast Serving of Retrieval Pipelines for Retrieval-Augmented Generationcited by 1
  36. 2026RANKVIDEO: Reasoning Reranking for Text-to-Video Retrievalcited by 1
  37. 2026Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coveragecited by 1
  38. 2026Test-time Scaling over Perception: Resolving the Grounding Paradox in Thinking with Imagescited by 1

50 most cited of 68 All 68 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Rerank Before You Reason: Analyzing Reranking Tradeoffs through Effective Token Cost in Deep Search Agents
  2. 2025Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learning
  3. 2026Stop Overthinking: Unlocking Efficient Listwise Reranking with Minimal Reasoning
  4. 2026Prism-Reranker: Beyond Relevance Scoring -- Jointly Producing Contributions and Evidence for Agentic Retrieval
  5. 2026Compute Allocation for Reasoning-Intensive Retrieval Agents
  6. 2026QReason: Query-Focused Decoupled Chain-of-Thought for Efficient Passage Reranking
  7. 2023Efficient Title Reranker for Fast and Improved Knowledge-Intense NLP
  8. 2025REARANK: Reasoning Re-ranking Agent via Reinforcement Learning