mlsanityData to 7 Oct 2026

Rank-K: Test-Time Reasoning for Listwise Reranking

arXiv 2505.14432 ↗First posted 20 May 2025cs.IR cs.CLcited by 28

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Retrieve-and-rerank is a popular retrieval pipeline because of its ability to make slow but effective rerankers efficient enough at query time by reducing the number of comparisons. Recent works in neural rerankers take advantage of large language models for their capability in reasoning between queries and passages and have achieved state-of-the-art retrieval effectiveness. However, such rerankers are resource-intensive, even after heavy optimization. In this work, we introduce Rank-K, a listwise passage reranking model that leverages the reasoning capability of the reasoning language model at query time that provides test time scalability to serve hard queries. We show that Rank-K improves retrieval effectiveness by 23% over the RankZephyr, the state-of-the-art listwise reranker, when reranking a BM25 initial ranked list and 19% when reranking strong retrieval results by SPLADE-v3. Since Rank-K is inherently a multilingual model, we found that it ranks passages based on queries in different languages as effectively as it does in monolingual retrieval.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 32 of its 67 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

32 of 67 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025ReasonIR: Training Retrievers for Reasoning Tasks29 Apr
  2. 2025Distillation and Refinement of Reasoning in Small Language Models for Document Re-ranking4 Apr
  3. 2025Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learning8 Mar
  4. 2025Rank1: Test-Time Compute for Reranking in Information Retrieval25 Feb

Other references · 28

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2023Mistral 7Bcited by 3,065
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  6. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  7. 2025s1: Simple test-time scalingcited by 1,311
  8. 2024OpenAI o1 System Cardcited by 1,220
  9. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  10. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  11. 2020Overview of the TREC 2019 deep learning trackcited by 282
  12. 2019Multi-Stage Document Ranking with BERTcited by 200
16 more
  1. 2021Overview of the TREC 2020 deep learning trackcited by 193
  2. 2024BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrievalcited by 153
  3. 2023RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!cited by 151
  4. 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 120
  5. 2021The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Modelscited by 81
  6. 2023Zero-Shot Listwise Document Reranking with a Large Language Modelcited by 81
  7. 2023RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Modelscited by 69
  8. 2024SPLADE-v3: New baselines for SPLADEcited by 69
  9. 2024FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructionscited by 67
  10. 2024Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Modelscited by 39
  11. 2023Scaling Down, LiTting Up: Efficient Zero-Shot Listwise Reranking with Seq2seq Encoder-Decoder Modelscited by 26
  12. 2025Is That Your Final Answer? Test-Time Scaling Improves Selective Question Answeringcited by 17
  13. 2022NELLIE: A Neuro-Symbolic Inference Engine for Grounded, Compositional, and Explainable Reasoningcited by 16
  14. 2024Enhancing Systematic Decompositional Natural Language Inference Using Informal Logiccited by 12
  15. 2023NeuralMind-UNICAMP at 2022 TREC NeuCLIR: Large Boring Rerankers for Cross-lingual Retrievalcited by 7
  16. 2025Variations in Relevance Judgments and the Shelf Life of Test Collectionscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 13; 2026: 15.

28 papers.

Sort:

By area:

  1. 2025DIVER: A Multi-Stage Approach for Reasoning-intensive Information Retrievalcited by 35
  2. 2025ReasonEmbed: Enhanced Text Embeddings for Reasoning-Intensive Document Retrievalcited by 15
  3. 2025ThinkQE: Query Expansion via an Evolving Thinking Processcited by 11
  4. 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
  5. 2025Retro*: Optimizing LLMs for Reasoning-Intensive Document Retrievalcited by 7
  6. 2025Rethinking Reasoning in Document Ranking: Why Chain-of-Thought Falls Shortcited by 7
  7. 2026Revisiting Text Ranking in Deep Researchcited by 7
  8. 2025TFRank: Think-Free Reasoning Enables Practical Pointwise LLM Rankingcited by 5
  9. 2025InsertRank: LLMs can reason over BM25 scores to Improve Listwise Rerankingcited by 4
  10. 2025Language Models and Logic Programs for Trustworthy Tax Reasoningcited by 4
  11. 2026Learning from Emptiness: De-biasing Listwise Rerankers with Content-Agnostic Probability Calibrationcited by 4
  12. 2025GroupRank: A Groupwise Paradigm for Effective and Efficient Passage Reranking with LLMscited by 3
16 more
  1. 2025An Index-based Approach for Efficient and Effective Web Content Extractioncited by 2
  2. 2026Does Reasoning Make Search More Fair? Comparing Fairness in Reasoning and Non-Reasoning Rerankerscited by 2
  3. 2026CoverageBench: Evaluating Information Coverage across Tasks and Domainscited by 2
  4. 2025Relevance to Utility: Process-Supervised Rewrite for RAGcited by 1
  5. 2025You Only Forward Once: An Efficient Compositional Judging Paradigmcited by 1
  6. 2026RANKVIDEO: Reasoning Reranking for Text-to-Video Retrievalcited by 1
  7. 2026A Survey of Reasoning-Intensive Retrieval: Progress and Challengescited by 1
  8. 2026Bringing Agentic Search to Earth Observation Data Discoverycited by 1
  9. 2025How Grounded is Wikipedia? A Study on Structured Evidential Support and Retrievalcited by 0
  10. 2026Re-Rankers as Relevance Judgescited by 0
  11. 2026CRE-T1 Preview Technical Report: Beyond Contrastive Learning for Reasoning-Intensive Retrievalcited by 0
  12. 2026DualView: Adaptive Local-Global Fusion for Multi-Hop Document Rerankingcited by 0
  13. 2026Beyond Polarization: The Generative Constraint of Chain-of-Thought in Pointwise Rerankingcited by 0
  14. 2026How Calibration Content Shapes Attention-Based Rerankingcited by 0
  15. 2026QReason: Query-Focused Decoupled Chain-of-Thought for Efficient Passage Rerankingcited by 0
  16. 2026Where Root Cause Analysis Fails: A Retrieval-Reranking Decompositioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Stop Overthinking: Unlocking Efficient Listwise Reranking with Minimal Reasoning
  2. 2026Think Thrice Before Reranking: Multi-perspective Evidence and Reasoning Integration for Text Reranking
  3. 2025ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability
  4. 2025ListConRanker: A Contrastive Text Reranker with Listwise Encoding
  5. 2024Leveraging Passage Embeddings for Efficient Listwise Reranking with Large Language Models
  6. 2025The Evolution of Reranking Models in Information Retrieval: From Heuristic Methods to Large Language Models
  7. 2025RankLLM: A Python Package for Reranking with LLMs
  8. 2026TabRank: Chain-of-Thought Distillation for Table Re-Rankers