# Ranked List Truncation for Large Language Model-based Re-Ranking

arXiv: https://arxiv.org/abs/2404.18185 · HTML: https://mlsanity.com/paper/2404.18185/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 63 references that are in the map.
- First posted on arXiv 28 Apr 2024.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://mlsanity.com/paper/2304.09542/) (2023)
- [Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting](https://mlsanity.com/paper/2306.17563/) (2023)
- [Overview of the TREC 2019 deep learning track](https://mlsanity.com/paper/2003.07820/) (2020)
- [Multi-Stage Document Ranking with BERT](https://mlsanity.com/paper/1910.14424/) (2019)
- [Overview of the TREC 2020 deep learning track](https://mlsanity.com/paper/2102.07662/) (2021)
- [Improving Passage Retrieval with Zero-Shot Question Generation](https://mlsanity.com/paper/2204.07496/) (2022)
- [From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective](https://mlsanity.com/paper/2205.04733/) (2022)
- [Beyond Yes and No: Improving Zero-Shot LLM Rankers via Scoring Fine-Grained Relevance Labels](https://mlsanity.com/paper/2310.14122/) (2023)
- [Efficient Document Re-Ranking for Transformers by Precomputing Term Representations](https://mlsanity.com/paper/2004.14255/) (2020)
- [RefNet: A Reference-aware Network for Background Based Conversation](https://mlsanity.com/paper/1908.06449/) (2019)
- [Adaptive Re-Ranking with a Corpus Graph](https://mlsanity.com/paper/2208.08942/) (2022)
- [Interpretable & Time-Budget-Constrained Contextualization for Re-Ranking](https://mlsanity.com/paper/2002.01854/) (2020)
- [The Cascade Transformer: an Application for Efficient Answer Sentence Selection](https://mlsanity.com/paper/2005.02534/) (2020)
- [Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Models](https://mlsanity.com/paper/2105.04651/) (2021)
- [Self-seeding and Multi-intent Self-instructing LLMs for Generating Intent-aware Information-Seeking dialogs](https://mlsanity.com/paper/2402.11633/) (2024)
- [Query Performance Prediction: From Ad-hoc to Conversational Search](https://mlsanity.com/paper/2305.10923/) (2023)
- [Understanding BERT Rankers Under Distillation](https://mlsanity.com/paper/2007.11088/) (2020)
- [A Modern Perspective on Query Likelihood with Deep Generative Retrieval Models](https://mlsanity.com/paper/2106.13618/) (2021)
- [Choppy: Cut Transformer For Ranked List Truncation](https://mlsanity.com/paper/2004.13012/) (2020)
- [Sparse Pairwise Re-ranking with Pre-trained Transformers](https://mlsanity.com/paper/2207.04470/) (2022)
- [Efficient and Effective Tree-based and Neural Learning to Rank](https://mlsanity.com/paper/2305.08680/) (2023)
- [Surprise: Result List Truncation via Extreme Value Theory](https://mlsanity.com/paper/2010.09797/) (2020)
- [Learning to Truncate Ranked Lists for Information Retrieval](https://mlsanity.com/paper/2102.12793/) (2021)
- [Certified Error Control of Candidate Set Pruning for Two-Stage Relevance Ranking](https://mlsanity.com/paper/2205.09638/) (2022)

## Built on by (most cited first, 20 of 20)

- [Human-LLM Compound System for Scientific Ideation through Facet Recombination and Novelty Evaluation](https://mlsanity.com/paper/2409.14634/) (2024)
- [A Survey of Large Language Model Empowered Agents for Recommendation and Search: Towards Next-Generation Information Retrieval](https://mlsanity.com/paper/2503.05659/) (2025)
- [Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers](https://mlsanity.com/paper/2506.00054/) (2025)
- [Literature-Grounded Novelty Assessment of Scientific Ideas](https://mlsanity.com/paper/2506.22026/) (2025)
- [Retrieval-Enhanced Machine Learning: Synthesis and Opportunities](https://mlsanity.com/paper/2407.12982/) (2024)
- [AcuRank: Uncertainty-Aware Adaptive Computation for Listwise Reranking](https://mlsanity.com/paper/2505.18512/) (2025)
- [Conversational Search: From Fundamentals to Frontiers in the LLM Era](https://mlsanity.com/paper/2506.10635/) (2025)
- [LLM Alignment as Retriever Optimization: An Information Retrieval Perspective](https://mlsanity.com/paper/2502.03699/) (2025)
- [Revisiting Text Ranking in Deep Research](https://mlsanity.com/paper/2602.21456/) (2026)
- [How Many Tools Should an LLM Agent See? A Chance-Corrected Answer](https://mlsanity.com/paper/2605.24660/) (2026)
- [The Next Phase of Scientific Fact-Checking: Advanced Evidence Retrieval from Complex Structured Academic Papers](https://mlsanity.com/paper/2506.20844/) (2025)
- [Zero-Shot and Efficient Clarification Need Prediction in Conversational Search](https://mlsanity.com/paper/2503.00179/) (2025)
- [Reinforcement Speculative Decoding for Fast Ranking](https://mlsanity.com/paper/2505.20316/) (2025)
- [Beyond Prompting: An Efficient Embedding Framework for Open-Domain Question Answering](https://mlsanity.com/paper/2503.01606/) (2025)
- [Optimizing Compound Retrieval Systems](https://mlsanity.com/paper/2504.12063/) (2025)
- [WXImpactBench: A Disruptive Weather Impact Understanding Benchmark for Evaluating Large Language Models](https://mlsanity.com/paper/2505.20249/) (2025)
- [heiDS at ArchEHR-QA 2025: From Fixed-k to Query-dependent-k for Retrieval Augmented Generation](https://mlsanity.com/paper/2506.19512/) (2025)
- [Learning to Route Queries to Heads for Attention-based Re-ranking with Large Language Models](https://mlsanity.com/paper/2604.24608/) (2026)
- [One Size Does Not Fit All! Dynamic Retriever and Generator Selection for RAG](https://mlsanity.com/paper/2609.17709/) (2026)
- [BoundaryMORPH: Budgeted Reranking via Active Set Selection for Diffuse Retrieval](https://mlsanity.com/paper/2609.27213/) (2026)

## Abstract (arXiv)

We study ranked list truncation (RLT) from a novel "retrieve-then-re-rank" perspective, where we optimize re-ranking by truncating the retrieved list (i.e., trim re-ranking candidates). RLT is crucial for re-ranking as it can improve re-ranking efficiency by sending variable-length candidate lists to a re-ranker on a per-query basis. It also has the potential to improve re-ranking effectiveness. Despite its importance, there is limited research into applying RLT methods to this new perspective. To address this research gap, we reproduce existing RLT methods in the context of re-ranking, especially newly emerged large language model (LLM)-based re-ranking. In particular, we examine to what extent established findings on RLT for retrieval are generalizable to the "retrieve-then-re-rank" setup from three perspectives: (i) assessing RLT methods in the context of LLM-based re-ranking with lexical first-stage retrieval, (ii) investigating the impact of different types of first-stage retrievers on RLT methods, and (iii) investigating the impact of different types of re-rankers on RLT methods. We perform experiments on the TREC 2019 and 2020 deep learning tracks, investigating 8 RLT methods for pipelines involving 3 retrievers and 2 re-rankers. We reach new insights into RLT methods in the context of re-ranking.