mlsanityData to 7 Oct 2026

Instruction Distillation Makes Large Language Models Efficient Zero-shot Rankers

arXiv 2311.01555 ↗First posted 2 Nov 2023cs.IR cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

Recent studies have demonstrated the great potential of Large Language Models (LLMs) serving as zero-shot relevance rankers. The typical approach involves making comparisons between pairs or lists of documents. Although effective, these listwise and pairwise methods are not efficient and also heavily rely on intricate prompt engineering. To tackle this problem, we introduce a novel instruction distillation method. The key idea is to distill the pairwise ranking ability of open-sourced LLMs to a simpler but more efficient pointwise ranking. Specifically, given the same LLM, we first rank documents using the effective pairwise approach with complex instructions, and then distill the teacher predictions to the pointwise approach with simpler instructions. Evaluation results on the BEIR, TREC, and ReDial datasets demonstrate that instruction distillation can improve efficiency by 10 to 100x and also enhance the ranking performance of LLMs. Furthermore, our approach surpasses the performance of existing supervised methods like monoT5 and is on par with the state-of-the-art zero-shot methods. The code to reproduce our results is available at www.github.com/sunnweiwei/RankGPT.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 27 of its 39 references that are in the map. First posted on arXiv 2 Nov 2023.

Builds on

27 of 39 references.

References · 27

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  3. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  4. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  5. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  6. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  7. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  8. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  9. 2023A Survey on Large Language Models for Recommendationcited by 378
  10. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  11. 2023TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendationcited by 336
  12. 2023Large Language Models for Information Retrieval: A Surveycited by 331
15 more
  1. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 307
  2. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 302
  3. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  4. 2022Teaching Small Language Models to Reasoncited by 279
  5. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 249
  6. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  7. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  8. 2021Overview of the TREC 2020 deep learning trackcited by 193
  9. 2018Towards Deep Conversational Recommendationscited by 174
  10. 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 164
  11. 2023Zero-Shot Listwise Document Reranking with a Large Language Modelcited by 81
  12. 2023Learning to Tokenize for Generative Retrievalcited by 66
  13. 2023Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Methodcited by 51
  14. 2022Questions Are All You Need to Train a Dense Passage Retrievercited by 47
  15. 2023Generative Knowledge Selection for Knowledge-Grounded Dialoguescited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 8; 2025: 6; 2026: 4.

19 papers.

Sort:

By area:

  1. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  2. 2025ReasonIR: Training Retrievers for Reasoning Taskscited by 76
  3. 2024TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategycited by 47
  4. 2024Efficient Prompting Methods for Large Language Models: A Surveycited by 46
  5. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  6. 2024Towards Completeness-Oriented Tool Retrieval for Large Language Modelscited by 26
  7. 2025Towards AI Search Paradigmcited by 19
  8. 2024DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Taskcited by 17
  9. 2025Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedbackcited by 12
  10. 2024MLLM as Retriever: Interactively Learning Multimodal Retrieval for Embodied Agentscited by 11
  11. 2023Distilling Rule-based Knowledge into Large Language Modelscited by 9
  12. 2025Knowledge Distillation for Enhancing Walmart E-commerce Search Relevance Using Large Language Modelscited by 4
7 more
  1. 2026Positive-Unlabeled Reinforcement Learning Distillation for On-Premise Small Modelscited by 4
  2. 2024EcoRank: Budget-Constrained Text Re-ranking Using Large Language Modelscited by 3
  3. 2024Best Practices for Distilling Large Language Models into BERT for Web Search Rankingcited by 3
  4. 2025LLM Optimization Unlocks Real-Time Pairwise Rerankingcited by 1
  5. 2026From Parameters to Behaviors: A Survey of Model Fusion for Large Language Modelscited by 1
  6. 2026ZooWork-ShopRanker: An Open, Preference-Aligned E-Commerce Rerankercited by 0
  7. 2026Distilling Agentic Systems: A Roadmap across Models, Artifacts, and Harnessescited by 0