mlsanityData to 7 Oct 2026

RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models

arXiv 2309.15088 ↗First posted 26 Sep 2023cs.IR cs.CLcited by 69

Not on the map. Why?

Abstract (arXiv)

Researchers have successfully applied large language models (LLMs) such as ChatGPT to reranking in an information retrieval context, but to date, such work has mostly been built on proprietary models hidden behind opaque API endpoints. This approach yields experimental results that are not reproducible and non-deterministic, threatening the veracity of outcomes that build on such shaky foundations. To address this significant shortcoming, we present RankVicuna, the first fully open-source LLM capable of performing high-quality listwise reranking in a zero-shot setting. Experimental results on the TREC 2019 and 2020 Deep Learning Tracks show that we can achieve effectiveness comparable to zero-shot reranking with GPT-3.5 with a much smaller 7B parameter model, although our effectiveness remains slightly behind reranking with GPT-4. We hope our work provides the foundation for future research on reranking with modern LLMs. All the code necessary to reproduce our results is available at https://github.com/castorini/rank_llm.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). Builds on 28 of its 41 references that are in the map. First posted on arXiv 26 Sep 2023.

Builds on

28 of 41 references.

References · 28

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  3. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  4. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  5. 2019Passage Re-ranking with BERTcited by 629
  6. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  7. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  8. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  9. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
  10. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 302
  11. 2020Overview of the TREC 2019 deep learning trackcited by 282
  12. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 266
16 more
  1. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  2. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  3. 2019Multi-Stage Document Ranking with BERTcited by 200
  4. 2021Overview of the TREC 2020 deep learning trackcited by 193
  5. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 126
  6. 2022RankT5: Fine-Tuning T5 for Text Ranking with Ranking Lossescited by 114
  7. 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
  8. 2021The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Modelscited by 81
  9. 2023Zero-Shot Listwise Document Reranking with a Large Language Modelcited by 81
  10. 2021Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipelinecited by 65
  11. 2022Questions Are All You Need to Train a Dense Passage Retrievercited by 47
  12. 2023How Does Generative Retrieval Scale to Millions of Passages?cited by 47
  13. 2021A Proposed Conceptual Framework for a Representational Approach to Information Retrievalcited by 24
  14. 2020Beyond [CLS] through Ranking by Generationcited by 20
  15. 2023InPars-Light: Cost-Effective Unsupervised Training of Efficient Rankerscited by 14
  16. 2023Vector Search with OpenAI Embeddings: Lucene Is All You Needcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 14; 2025: 27; 2026: 25.

69 papers.

Sort:

By area:

  1. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  2. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  3. 2024Generative Representational Instruction Tuningcited by 227
  4. 2025Rank1: Test-Time Compute for Reranking in Information Retrievalcited by 68
  5. 2023A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Modelscited by 53
  6. 2024Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankerscited by 42
  7. 2025ReasonRank: Empowering Passage Ranking with Strong Reasoning Abilitycited by 39
  8. 2024Ragnar"ok: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Trackcited by 31
  9. 2023Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Modelscited by 30
  10. 2025Rank-K: Test-Time Reasoning for Listwise Rerankingcited by 28
  11. 2025REARANK: Reasoning Re-ranking Agent via Reinforcement Learningcited by 24
  12. 2025jina-reranker-v3: Last but Not Late Interaction for Listwise Document Rerankingcited by 23
38 more
  1. 2025SweRank: Software Issue Localization with Code Rankingcited by 20
  2. 2025Guiding Retrieval using LLM-based Listwise Rankerscited by 16
  3. 2023Rank-without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Modelscited by 14
  4. 2025DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generationcited by 14
  5. 2024LitLLMs, LLMs for Literature Review: Are we there yet?cited by 12
  6. 2025Iterative Self-Incentivization Empowers Large Language Models as Agentic Searcherscited by 11
  7. 2025DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillationcited by 11
  8. 2025SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQLcited by 10
  9. 2024ReasoningRank: Teaching Student Models to Rank through Reasoning-Based Knowledge Distillationcited by 9
  10. 2025In-context Ranking Preference Optimizationcited by 9
  11. 2024rerankers: A Lightweight Python Library to Unify Ranking Methodscited by 8
  12. 2025Batched Self-Consistency Improves LLM Relevance Assessment and Rankingcited by 8
  13. 2025AcuRank: Uncertainty-Aware Adaptive Computation for Listwise Rerankingcited by 8
  14. 2025Scalable In-context Ranking with Generative Modelscited by 8
  15. 2024MCRanker: Generating Diverse Criteria On-the-Fly to Improve Point-wise LLM Rankerscited by 7
  16. 2024An Investigation of Prompt Variations for Zero-shot LLM-based Rankerscited by 7
  17. 2025Rethinking Reasoning in Document Ranking: Why Chain-of-Thought Falls Shortcited by 7
  18. 2024Squid: Long Context as a New Modality for Energy-Efficient On-Device Language Modelscited by 6
  19. 2025Distillation and Refinement of Reasoning in Small Language Models for Document Re-rankingcited by 6
  20. 2025Beyond Sequential Reranking: Reranker-Guided Search Improves Reasoning Intensive Retrievalcited by 5
  21. 2025A Survey of LLM × DATAcited by 4
  22. 2025Reinforcement Speculative Decoding for Fast Rankingcited by 3
  23. 2025The Evolution of Reranking Models in Information Retrieval: From Heuristic Methods to Large Language Modelscited by 3
  24. 2026SCI-Defense: Defending Manipulation Attacks from Generative Engine Optimizationcited by 3
  25. 2024Enhancing Exploratory Learning through Exploratory Search with the Emergence of Large Language Modelscited by 2
  26. 2025Beyond Retrieval: Ensembling Cross-Encoders and GPT Rerankers with LLMs for Biomedical QAcited by 2
  27. 2026Beyond Relevance-Centric Retrieval: Rubric-Oriented Document Set Selection and Rankingcited by 2
  28. 2024Large Language Models for Stemming: Promises, Pitfalls and Failurescited by 1
  29. 2024PyTerrier-GenRank: The PyTerrier Plugin for Reranking with Large Language Modelscited by 1
  30. 2025Access Paths for Efficient Ordering with Large Language Modelscited by 1
  31. 2025What Drives Cross-lingual Ranking? Retrieval Approaches with Multilingual Language Modelscited by 1
  32. 2026Autoregressive Ranking: Bridging the Gap Between Dual and Cross Encoderscited by 1
  33. 2026Offline Reasoning for Efficient Recommendation: LLM-Empowered Persona-Profiled Item Indexingcited by 1
  34. 2026Prism-Reranker: Beyond Relevance Scoring -- Jointly Producing Contributions and Evidence for Agentic Retrievalcited by 1
  35. 2026CAR: Query-Guided Confidence-Aware Reranking for Retrieval-Augmented Generationcited by 1
  36. 2026MagicSelector: Joint Optimization for Agent Tool Selection via Counterfactual Decomposition and Progressive Rerankingcited by 1
  37. 2024RaCT: Ranking-aware Chain-of-Thought Optimization for LLMscited by 0
  38. 2025Commonsense Generation and Evaluation for Dialogue Systems using Large Language Modelscited by 0

50 most cited of 69 All 69 (JSON)