mlsanityData to 7 Oct 2026

Few-shot Reranking for Multi-hop QA via Language Model Prompting

arXiv 2205.12650 ↗First posted 25 May 2022cs.CL cs.IRcited by 19

Not on the map. Why?

Abstract (arXiv)

We study few-shot reranking for multi-hop QA with open-domain questions. To alleviate the need for a large number of labeled question-document pairs for retriever training, we propose PromptRank, which relies on large language models prompting for multi-hop path reranking. PromptRank first constructs an instruction-based prompt that includes a candidate document path and then computes the relevance score between a given question and the path based on the conditional likelihood of the question given the path prompt according to a language model. PromptRank yields strong retrieval performance on HotpotQA with only 128 training examples compared to state-of-the-art methods trained on thousands of examples -- 73.6 recall@10 by PromptRank vs. 77.8 by PathRetriever and 77.5 by multi-hop dense retrieval. Code available at https://github.com/mukhal/PromptRank

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 40 of its 54 references that are in the map. First posted on arXiv 25 May 2022.

Builds on

40 of 54 references.

References · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2017Billion-scale similarity search with GPUscited by 2,477
  11. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  12. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
28 more
  1. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  2. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  3. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  4. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  5. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  6. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  7. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  8. 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 359
  9. 2020Calibration of Pre-trained Transformerscited by 307
  10. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 256
  11. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  12. 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 164
  13. 2020Answering Complex Open-Domain Questions with Multi-Hop Dense Retrievalcited by 160
  14. 2019Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answeringcited by 147
  15. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
  16. 2020Unsupervised Question Decomposition for Question Answeringcited by 118
  17. 2020RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphscited by 117
  18. 2020Few-Shot Text Generation with Pattern-Exploiting Trainingcited by 87
  19. 2019Revealing the Importance of Semantic Retrieval for Machine Reading at Scalecited by 50
  20. 2021Learning to Retrieve Passages without Supervisioncited by 46
  21. 2020Answering Open-Domain Questions of Varying Reasoning Steps from Textcited by 43
  22. 2020Differentiable Reasoning over a Virtual Knowledge Basecited by 39
  23. 2021Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrievalcited by 25
  24. 2018Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answeringcited by 24
  25. 2020Beyond [CLS] through Ranking by Generationcited by 20
  26. 2020HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questionscited by 19
  27. 2018Training a Ranking Function for Open-Domain Question Answeringcited by 4
  28. 2021Weakly Supervised Pre-Training for Multi-Hop Retrievercited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 10; 2025: 4; 2026: 2.

19 papers.

Sort:

By area:

  1. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  2. 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 104
  3. 2023PokeMQA: Programmable knowledge editing for Multi-hop Question Answeringcited by 23
  4. 2024ChatGPT Rates Natural Language Explanation Quality Like Humans: But on Which Scales?cited by 17
  5. 2024Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Enginescited by 14
  6. 2024Learning to Reason via Program Generation, Emulation, and Searchcited by 9
  7. 2025ASRank: Zero-Shot Re-Ranking with Answer Scent for Document Retrievalcited by 9
  8. 2025SGIC: A Self-Guided Iterative Calibration Framework for RAGcited by 8
  9. 2023Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learningcited by 6
  10. 2024Prompting Explicit and Implicit Knowledge for Multi-hop Question Answering Based on Human Reading Processcited by 4
  11. 2023Automating construction contract review using knowledge graph-enhanced large language modelscited by 3
  12. 2024Improving Retrieval Augmented Open-Domain Question-Answering with Vectorized Contextscited by 3
7 more
  1. 2024Red Teaming Language Models for Processing Contradictory Dialoguescited by 3
  2. 2024GenSco: Can Question Decomposition based Passage Alignment improve Question Answering?cited by 3
  3. 2024LightPAL: Lightweight Passage Retrieval for Open Domain Multi-Document Summarizationcited by 1
  4. 2025Learning to Extract Rational Evidence via Reinforcement Learning for Retrieval-Augmented Generationcited by 1
  5. 2026STRIDE: Strategic Iterative Decision-Making for Retrieval-Augmented Multi-Hop Question Answeringcited by 1
  6. 2025DMA: Online RAG Alignment with Human Feedbackcited by 0
  7. 2026Evaluating Prompt Engineering Techniques for RAG in Small Language Models: A Multi-Hop QA Approachcited by 0