mlsanityData to 14 Oct 2026

LLM4Ranking: An Easy-to-use Framework of Utilizing Large Language Models for Document Reranking

arXiv 2504.07439 ↗First posted 10 Apr 2025cs.IR cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Utilizing large language models (LLMs) for document reranking has been a popular and promising research direction in recent years, many studies are dedicated to improving the performance and efficiency of using LLMs for reranking. Besides, it can also be applied in many real-world applications, such as search engines or retrieval-augmented generation. In response to the growing demand for research and application in practice, we introduce a unified framework, LLM4Ranking, which enables users to adopt different ranking methods using open-source or closed-source API-based LLMs. Our framework provides a simple and extensible interface for document reranking with LLMs, as well as easy-to-use evaluation and fine-tuning scripts for this task. We conducted experiments based on this framework and evaluated various models and methods on several widely used datasets, providing reproducibility results on utilizing LLMs for document reranking. Our code is publicly available at https://github.com/liuqi6777/llm4ranking.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 26 of its 36 references that are in the map. First posted on arXiv 10 Apr 2025.

Builds on

26 of 36 references.

References · 26

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  6. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  7. 2019Passage Re-ranking with BERTcited by 645
  8. 20218-bit Optimizers via Block-wise Quantizationcited by 403
  9. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 360
  10. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  11. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 307
  12. 2020Overview of the TREC 2019 deep learning trackcited by 286
14 more
  1. 2019Document Expansion by Query Predictioncited by 231
  2. 2024BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrievalcited by 158
  3. 2023RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!cited by 155
  4. 2023Beyond Yes and No: Improving Zero-Shot LLM Rankers via Scoring Fine-Grained Relevance Labelscited by 86
  5. 2021The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Modelscited by 81
  6. 2024FIRST: Faster Improved Listwise Reranking with Single Token Decodingcited by 54
  7. 2022Tevatron: An Efficient and Flexible Toolkit for Dense Retrievalcited by 47
  8. 2022Questions Are All You Need to Train a Dense Passage Retrievercited by 47
  9. 2024TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategycited by 47
  10. 2024Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankerscited by 43
  11. 2023NevIR: Negation in Neural Information Retrievalcited by 29
  12. 2024MAIR: A Massive Benchmark for Evaluating Instructed Retrievalcited by 23
  13. 2024DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Taskcited by 17
  14. 2024rerankers: A Lightweight Python Library to Unify Ranking Methodscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 7.

12 papers.

  1. 2025mRAG: Elucidating the Design Space of Multi-modal Retrieval-Augmented Generationcited by 4
  2. 2025E2Rank: Unifying Text Embedding and Listwise Reranking for Effective and Efficient Searchcited by 4
  3. 2026RLPO: Residual Listwise Preference Optimization for Long-Context Review Rankingcited by 3
  4. 2026DiffuRank: Effective Document Reranking with Diffusion Language Modelscited by 3
  5. 2026SearchArt: Training Long-Horizon Search Agent with Scalable Synthetic and Verified Taskcited by 3
  6. 2025Mitigating Hallucination in Large Language Models (LLMs): An Application-Oriented Survey on RAG, Reasoning, and Agentic Systemscited by 2
  7. 2025R1-Ranker: Teaching LLM Rankers to Reasoncited by 1
  8. 2026RoutIR: Fast Serving of Retrieval Pipelines for Retrieval-Augmented Generationcited by 1
  9. 2025How Reliable are LLMs for Reasoning on the Re-ranking task?cited by 0
  10. 2026Scaling Search Relevance: Augmenting App Store Ranking with LLM-Generated Judgmentscited by 0
  11. 2026DocRetriever: A Plug-and-Play Framework for Multimodal Document Retrieval with Comprehensive Benchmarkcited by 0
  12. 2026PRISMR: Overcoming Parse Collapse in Multimodal Listwise Ranking via Parameterized Representation Internalizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RankLLM: A Python Package for Reranking with LLMs
  2. 2024LLM4Rerank: LLM-based Auto-Reranking Framework for Recommendations
  3. 2025InsertRank: LLMs can reason over BM25 scores to Improve Listwise Reranking
  4. 2026LLM-Confidence Reranker: A Training-Free Approach for Enhancing Retrieval-Augmented Generation Systems
  5. 2025LLM Optimization Unlocks Real-Time Pairwise Reranking
  6. 2025The Evolution of Reranking Models in Information Retrieval: From Heuristic Methods to Large Language Models
  7. 2025EnrichIndex: Using LLMs to Enrich Retrieval Indices Offline
  8. 2025GroupRank: A Groupwise Paradigm for Effective and Efficient Passage Reranking with LLMs