# Sliding Windows Are Not the End: Exploring Full Ranking with Long-Context Large Language Models

arXiv: https://arxiv.org/abs/2412.14574 · HTML: https://mlsanity.com/paper/2412.14574/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Dec 2024.

## Built on by (most cited first, 8 of 8)

- [Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-ranking](https://mlsanity.com/paper/2506.09944/) (2025)
- [Towards AI Search Paradigm](https://mlsanity.com/paper/2506.17188/) (2025)
- [Agentic-R: Learning to Retrieve for Agentic Search](https://mlsanity.com/paper/2601.11888/) (2026)
- [CoRank: LLM-Based Compact Reranking with Document Features for Scientific Retrieval](https://mlsanity.com/paper/2505.13757/) (2025)
- [Beyond Relevance-Centric Retrieval: Rubric-Oriented Document Set Selection and Ranking](https://mlsanity.com/paper/2607.19747/) (2026)
- [Benchmarking Chinese Medical LLMs: A Medbench-based Analysis of Performance Gaps and Hierarchical Optimization Strategies](https://mlsanity.com/paper/2503.07306/) (2025)
- [Can LLM Rerankers Predict Their Own Ranking Performance?](https://mlsanity.com/paper/2606.03535/) (2026)
- [ResRank: Unifying Retrieval and Listwise Reranking via End-to-End Joint Training with Residual Passage Compression](https://mlsanity.com/paper/2604.22180/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have shown exciting performance in listwise passage ranking. Due to the limited input length, existing methods often adopt the sliding window strategy. Such a strategy, though effective, is inefficient as it involves repetitive and serialized processing, which usually re-evaluates relevant passages multiple times. As a result, it incurs redundant API costs, which are proportional to the number of inference tokens. The development of long-context LLMs enables the full ranking of all passages within a single inference, avoiding redundant API costs. In this paper, we conduct a comprehensive study of long-context LLMs for ranking tasks in terms of efficiency and effectiveness. Surprisingly, our experiments reveal that full ranking with long-context LLMs can deliver superior performance in the supervised fine-tuning setting with a huge efficiency improvement. Furthermore, we identify two limitations of fine-tuning the full ranking model based on existing methods: (1) sliding window strategy fails to produce a full ranking list as a training label, and (2) the language modeling loss cannot emphasize top-ranked passage IDs in the label. To alleviate these issues, we propose a new complete listwise label construction approach and a novel importance-aware learning objective for full ranking. Experiments show the superior performance of our method over baselines. Our codes are available at urlhttps://github.com/8421BCD/fullrank.