# RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models

arXiv: https://arxiv.org/abs/2309.15088 · HTML: https://mlsanity.com/paper/2309.15088/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 41 references that are in the map.
- First posted on arXiv 26 Sep 2023.

## Builds on (most cited first, 28 of 28)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [Text and Code Embeddings by Contrastive Pre-Training](https://mlsanity.com/paper/2201.10005/) (2022)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://mlsanity.com/paper/2304.09542/) (2023)
- [Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting](https://mlsanity.com/paper/2306.17563/) (2023)
- [Overview of the TREC 2019 deep learning track](https://mlsanity.com/paper/2003.07820/) (2020)
- [Pretrained Transformers for Text Ranking: BERT and Beyond](https://mlsanity.com/paper/2010.06467/) (2020)
- [Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling](https://mlsanity.com/paper/2104.06967/) (2021)
- [Promptagator: Few-shot Dense Retrieval From 8 Examples](https://mlsanity.com/paper/2209.11755/) (2022)
- [Multi-Stage Document Ranking with BERT](https://mlsanity.com/paper/1910.14424/) (2019)
- [Overview of the TREC 2020 deep learning track](https://mlsanity.com/paper/2102.07662/) (2021)
- [SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval](https://mlsanity.com/paper/2109.10086/) (2021)
- [RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses](https://mlsanity.com/paper/2210.10634/) (2022)
- [From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective](https://mlsanity.com/paper/2205.04733/) (2022)
- [Zero-Shot Listwise Document Reranking with a Large Language Model](https://mlsanity.com/paper/2305.02156/) (2023)
- [The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models](https://mlsanity.com/paper/2101.05667/) (2021)
- [Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipeline](https://mlsanity.com/paper/2101.08751/) (2021)
- [How Does Generative Retrieval Scale to Millions of Passages?](https://mlsanity.com/paper/2305.11841/) (2023)
- [Questions Are All You Need to Train a Dense Passage Retriever](https://mlsanity.com/paper/2206.10658/) (2022)
- [A Proposed Conceptual Framework for a Representational Approach to Information Retrieval](https://mlsanity.com/paper/2110.01529/) (2021)
- [Beyond [CLS] through Ranking by Generation](https://mlsanity.com/paper/2010.03073/) (2020)
- [InPars-Light: Cost-Effective Unsupervised Training of Efficient Rankers](https://mlsanity.com/paper/2301.02998/) (2023)
- [Vector Search with OpenAI Embeddings: Lucene Is All You Need](https://mlsanity.com/paper/2308.14963/) (2023)

## Built on by (most cited first, 30 of 70)

- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [A Survey on Knowledge Distillation of Large Language Models](https://mlsanity.com/paper/2402.13116/) (2024)
- [Generative Representational Instruction Tuning](https://mlsanity.com/paper/2402.09906/) (2024)
- [Rank1: Test-Time Compute for Reranking in Information Retrieval](https://mlsanity.com/paper/2502.18418/) (2025)
- [A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Models](https://mlsanity.com/paper/2310.09497/) (2023)
- [Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankers](https://mlsanity.com/paper/2410.02642/) (2024)
- [ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability](https://mlsanity.com/paper/2508.07050/) (2025)
- [Ragnar"ok: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Track](https://mlsanity.com/paper/2406.16828/) (2024)
- [Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Models](https://mlsanity.com/paper/2310.07712/) (2023)
- [Rank-K: Test-Time Reasoning for Listwise Reranking](https://mlsanity.com/paper/2505.14432/) (2025)
- [REARANK: Reasoning Re-ranking Agent via Reinforcement Learning](https://mlsanity.com/paper/2505.20046/) (2025)
- [jina-reranker-v3: Last but Not Late Interaction for Listwise Document Reranking](https://mlsanity.com/paper/2509.25085/) (2025)
- [SweRank: Software Issue Localization with Code Ranking](https://mlsanity.com/paper/2505.07849/) (2025)
- [Guiding Retrieval using LLM-based Listwise Rankers](https://mlsanity.com/paper/2501.09186/) (2025)
- [DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.07233/) (2025)
- [Rank-without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Models](https://mlsanity.com/paper/2312.02969/) (2023)
- [LitLLMs, LLMs for Literature Review: Are we there yet?](https://mlsanity.com/paper/2412.15249/) (2024)
- [SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQL](https://mlsanity.com/paper/2506.04494/) (2025)
- [Iterative Self-Incentivization Empowers Large Language Models as Agentic Searchers](https://mlsanity.com/paper/2505.20128/) (2025)
- [DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillation](https://mlsanity.com/paper/2508.16998/) (2025)
- [ReasoningRank: Teaching Student Models to Rank through Reasoning-Based Knowledge Distillation](https://mlsanity.com/paper/2410.05168/) (2024)
- [In-context Ranking Preference Optimization](https://mlsanity.com/paper/2504.15477/) (2025)
- [An Investigation of Prompt Variations for Zero-shot LLM-based Rankers](https://mlsanity.com/paper/2406.14117/) (2024)
- [rerankers: A Lightweight Python Library to Unify Ranking Methods](https://mlsanity.com/paper/2408.17344/) (2024)
- [Batched Self-Consistency Improves LLM Relevance Assessment and Ranking](https://mlsanity.com/paper/2505.12570/) (2025)
- [AcuRank: Uncertainty-Aware Adaptive Computation for Listwise Reranking](https://mlsanity.com/paper/2505.18512/) (2025)
- [Scalable In-context Ranking with Generative Models](https://mlsanity.com/paper/2510.05396/) (2025)
- [MCRanker: Generating Diverse Criteria On-the-Fly to Improve Point-wise LLM Rankers](https://mlsanity.com/paper/2404.11960/) (2024)
- [Rethinking Reasoning in Document Ranking: Why Chain-of-Thought Falls Short](https://mlsanity.com/paper/2510.08985/) (2025)
- [Squid: Long Context as a New Modality for Energy-Efficient On-Device Language Models](https://mlsanity.com/paper/2408.15518/) (2024)

## Abstract (arXiv)

Researchers have successfully applied large language models (LLMs) such as ChatGPT to reranking in an information retrieval context, but to date, such work has mostly been built on proprietary models hidden behind opaque API endpoints. This approach yields experimental results that are not reproducible and non-deterministic, threatening the veracity of outcomes that build on such shaky foundations. To address this significant shortcoming, we present RankVicuna, the first fully open-source LLM capable of performing high-quality listwise reranking in a zero-shot setting. Experimental results on the TREC 2019 and 2020 Deep Learning Tracks show that we can achieve effectiveness comparable to zero-shot reranking with GPT-3.5 with a much smaller 7B parameter model, although our effectiveness remains slightly behind reranking with GPT-4. We hope our work provides the foundation for future research on reranking with modern LLMs. All the code necessary to reproduce our results is available at https://github.com/castorini/rank_llm.