mlsanityData to 7 Oct 2026

Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking

arXiv 2310.13243 ↗First posted 20 Oct 2023cs.IR cs.CLcited by 44

Not on the map. Why?

Abstract (arXiv)

In the field of information retrieval, Query Likelihood Models (QLMs) rank documents based on the probability of generating the query given the content of a document. Recently, advanced large language models (LLMs) have emerged as effective QLMs, showcasing promising ranking capabilities. This paper focuses on investigating the genuine zero-shot ranking effectiveness of recent LLMs, which are solely pre-trained on unstructured text data without supervised instruction fine-tuning. Our findings reveal the robust zero-shot ranking ability of such LLMs, highlighting that additional instruction fine-tuning may hinder effectiveness unless a question generation task is present in the fine-tuning dataset. Furthermore, we introduce a novel state-of-the-art ranking system that integrates LLM-based QLMs with a hybrid zero-shot retriever, demonstrating exceptional effectiveness in both zero-shot and few-shot scenarios. We make our codebase publicly available at https://github.com/ielab/llm-qlm.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 28 of its 50 references that are in the map. First posted on arXiv 20 Oct 2023.

Builds on

28 of 50 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  8. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  9. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  10. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  11. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  12. 2019Passage Re-ranking with BERTcited by 629
16 more
  1. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  2. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  3. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 283
  4. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 266
  5. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  6. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  7. 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 164
  8. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 116
  9. 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
  10. 2022SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrievalcited by 90
  11. 2023InPars-v2: Large Language Models as Efficient Dataset Generators for Information Retrievalcited by 73
  12. 2021A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniquescited by 69
  13. 2021Dealing with Typos for BERT-based Passage Retrieval and Rankingcited by 23
  14. 2022CharacterBERT and Self-Teaching for Improving the Robustness of Dense Retrievers on Queries with Typoscited by 22
  15. 2021Fast Passage Re-ranking with Contextualized Exact Term Matching and Efficient Passage Expansioncited by 20
  16. 2021A Modern Perspective on Query Likelihood with Deep Generative Retrieval Modelscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 24; 2025: 13; 2026: 6.

44 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2024Language agents achieve superhuman synthesis of scientific knowledgecited by 121
  3. 2025Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learningcited by 58
  4. 2024LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answeringcited by 49
  5. 2025A Survey of Large Language Model Agents for Question Answeringcited by 46
  6. 2024Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankerscited by 42
  7. 2024M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitionscited by 38
  8. 2024CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Rerankingcited by 37
  9. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  10. 2024BAdam: A Memory Efficient Full Parameter Optimization Method for Large Language Modelscited by 27
  11. 2024Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generationcited by 27
  12. 2024A Survey of Conversational Searchcited by 22
32 more
  1. 2025MM-R5: MultiModal Reasoning-Enhanced ReRanker via Reinforcement Learning for Document Retrievalcited by 21
  2. 2024INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuningcited by 19
  3. 2024Query Performance Prediction using Relevance Judgments Generated by Large Language Modelscited by 18
  4. 2024Do Large Language Models Rank Fairly? An Empirical Study on the Fairness of LLMs as Rankerscited by 17
  5. 2024DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Taskcited by 17
  6. 2024Planning in the Dark: LLM-Symbolic Planning Pipeline without Expertscited by 15
  7. 2025ArgRAG: Explainable Retrieval Augmented Generation using Quantitative Bipolar Argumentationcited by 12
  8. 2025ProRank: Prompt Warmup via Reinforcement Learning for Small Language Models Rerankingcited by 11
  9. 2024No Free Lunch: Retrieval-Augmented Generation Undermines Fairness in LLMs, Even for Vigilant Userscited by 9
  10. 2025Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generationcited by 9
  11. 2025TeleRAG: Efficient Retrieval-Augmented Generation Inference with Lookahead Retrievalcited by 9
  12. 2024ReSLLM: Large Language Models are Strong Resource Selectors for Federated Searchcited by 7
  13. 2024W-RAG: Weakly Supervised Dense Retrieval in RAG for Open-domain Question Answeringcited by 6
  14. 2024Q-PEFT: Query-dependent Parameter Efficient Fine-tuning for Text Reranking with Large Language Modelscited by 4
  15. 2024Save It All: Enabling Full Parameter Tuning for Federated Large Language Models via Cycle Block Gradient Descentcited by 4
  16. 2025FedRAG: A Framework for Fine-Tuning Retrieval-Augmented Generation Systemscited by 4
  17. 2026DF-RAG: Query-Aware Diversity for Retrieval-Augmented Generationcited by 4
  18. 2024Improving Zero-shot LLM Re-Ranker with Risk Minimizationcited by 3
  19. 2024Multi-Conditional Ranking with Large Language Modelscited by 2
  20. 2024Passage-specific Prompt Tuning for Passage Reranking in Question Answering with Large Language Modelscited by 2
  21. 2024HRDE: Retrieval-Augmented Large Language Models for Chinese Health Rumor Detection and Explainabilitycited by 2
  22. 2024SEGMENT+: Long Text Processing with Short-Context Language Modelscited by 2
  23. 2025Better Benchmarking LLMs for Zero-Shot Dependency Parsingcited by 2
  24. 2025LLM-VPRF: Large Language Model Based Vector Pseudo Relevance Feedbackcited by 2
  25. 2026Legal-DC: Benchmarking Retrieval-Augmented Generation for Legal Documentscited by 2
  26. 2026HeadRank: Decoding-Free Passage Reranking via Preference-Aligned Attention Headscited by 2
  27. 2024Improving Generated and Retrieved Knowledge Combination Through Zero-shot Generationcited by 1
  28. 2025Addressing the sustainable AI trilemma: a case study on LLM agents and RAGcited by 1
  29. 2025What's in a Query: Polarity-Aware Distribution-Based Fair Rankingcited by 1
  30. 2026The Router Within: Eliciting Native Skill Routing from a Frozen LLMcited by 1
  31. 2026Attribution Techniques for Mitigating Hallucinated Information in RAG Systems: A Surveycited by 0
  32. 2026Multimodal LLM-Empowered Re-Ranking for Generalizable Person Re-Identificationcited by 0