# LTRR: Learning To Rank Retrievers for LLMs

arXiv: https://arxiv.org/abs/2506.13743 · HTML: https://mlsanity.com/paper/2506.13743/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 46 references that are in the map.
- First posted on arXiv 16 Jun 2025.

## Builds on (most cited first, 19 of 19)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://mlsanity.com/paper/2406.17557/) (2024)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity](https://mlsanity.com/paper/2403.14403/) (2024)
- [ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning](https://mlsanity.com/paper/2503.19470/) (2025)
- [Evaluating Retrieval Quality in Retrieval-Augmented Generation](https://mlsanity.com/paper/2404.13781/) (2024)
- [Tomayto, Tomahto. Beyond Token-level Answer Equivalence for Question Answering Evaluation](https://mlsanity.com/paper/2202.07654/) (2022)
- [Evaluating Stochastic Rankings with Expected Exposure](https://mlsanity.com/paper/2004.13157/) (2020)
- [Retrieval-Enhanced Machine Learning](https://mlsanity.com/paper/2205.01230/) (2022)
- [Computationally Efficient Optimization of Plackett-Luce Ranking Models for Relevance and Fairness](https://mlsanity.com/paper/2105.00855/) (2021)
- [Leveraging LLMs for Unsupervised Dense Retriever Ranking](https://mlsanity.com/paper/2402.04853/) (2024)
- [Towards Fair RAG: On the Impact of Fair Ranking in Retrieval-Augmented Generation](https://mlsanity.com/paper/2409.11598/) (2024)
- [Generating Diverse Q&A Benchmarks for RAG Evaluation with DataMorgana](https://mlsanity.com/paper/2501.12789/) (2025)
- [RouterRetriever: Routing over a Mixture of Expert Embedding Models](https://mlsanity.com/paper/2409.02685/) (2024)
- [Query Routing for Homogeneous Tools: An Instantiation in the RAG Scenario](https://mlsanity.com/paper/2406.12429/) (2024)
- [Adapting to Non-Stationary Environments: Multi-Armed Bandit Enhanced Retrieval-Augmented Generation on Knowledge Graphs](https://mlsanity.com/paper/2412.07618/) (2024)
- [Selecting which Dense Retriever to use for Zero-Shot Search](https://mlsanity.com/paper/2309.09403/) (2023)

## Built on by (most cited first, 4 of 4)

- [MoR: Better Handling Diverse Queries with a Mixture of Sparse, Dense, and Human Retrievers](https://mlsanity.com/paper/2506.15862/) (2025)
- [Evaluation of Agents under Simulated AI Marketplace Dynamics](https://mlsanity.com/paper/2604.14256/) (2026)
- [SIGIR 2025 -- LiveRAG Challenge Report](https://mlsanity.com/paper/2507.04942/) (2025)
- [When Context Misleads: Intent-Guided Decoding for Robust Retrieval-Augmented Generation](https://mlsanity.com/paper/2608.16515/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Generation (RAG) systems typically rely on a single fixed retriever, despite growing evidence that no single retriever performs optimally across all query types. In this paper, we explore a query routing approach that dynamically selects from a pool of retrievers based on the query, using both train-free heuristics and learned routing models. We frame routing as a learning-to-rank problem and introduce LTRR, a framework that Learns To Rank Retrievers according to their expected contribution to downstream RAG performance. Through experiments on diverse question-answering benchmarks with controlled variations in query types, we demonstrate that routing-based RAG consistently surpasses the strongest single-retriever baselines. The gains are particularly substantial when training with the Answer Correctness (AC) objective and when using pairwise ranking methods, with XGBoost yielding the best results. Additionally, our approach exhibits stronger generalization to out-of-distribution queries. Overall, our results underscore the critical role of both training strategy and optimization metric choice in effective query routing for RAG systems.