mlsanityData to 7 Oct 2026

RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs

arXiv 2407.02485 ↗First posted 2 Jul 2024cs.CL cs.AI cs.IRcited by 200

On the map: Retrieval-augmented generation → Generation and context integration

Abstract (arXiv)

Large language models (LLMs) typically utilize the top-k contexts from a retriever in retrieval-augmented generation (RAG). In this work, we propose a novel instruction fine-tuning framework RankRAG, which instruction-tunes a single LLM for the dual purpose of context ranking and answer generation in RAG. In particular, the instruction-tuned LLMs work surprisingly well by adding a small fraction of ranking data into the training blend, and outperform existing expert ranking models, including the same LLM exclusively fine-tuned on a large amount of ranking data. For generation, we compare our model with many strong baselines, including GPT-4-0613, GPT-4-turbo-2024-0409, and ChatQA-1.5, an open-sourced model with the state-of-the-art performance on RAG benchmarks. Specifically, our Llama3-RankRAG significantly outperforms Llama3-ChatQA-1.5 and GPT-4 models on nine knowledge-intensive benchmarks. In addition, it also performs comparably to GPT-4 on five RAG benchmarks in the biomedical domain without instruction fine-tuning on biomedical data, demonstrating its superb capability for generalization to new domains.

Key facts. Cited by 200 papers in the map (data through 7 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 82 of its 98 references that are in the map. First posted on arXiv 2 Jul 2024.

Builds on

82 of 98 references.

Closest ancestors · 13

Same direction, newest first

  1. 2024Reliable, Adaptable, and Attributable Language Models with Retrieval5 Mar
  2. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models15 Nov
  3. 2023InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining11 Oct
  4. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Context2 Oct
  5. 2023RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models15 Aug
  6. 2023Active Retrieval Augmented Generation11 May
  7. 2023In-Context Retrieval-Augmented Language Models31 Jan
  8. 2023REPLUG: Retrieval-Augmented Black-Box Language Models30 Jan
  9. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP28 Dec
  10. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories20 Dec
  11. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Models5 Aug
  12. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May
1 more
  1. 2020REALM: Retrieval-Augmented Language Model Pre-Training10 Feb

Other references · 69

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  11. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  12. 2024Mixtral of Expertscited by 1,785
38 more
  1. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  2. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  3. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  4. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  5. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  6. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  7. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  8. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  9. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  10. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  11. 2023PaLM 2 Technical Reportcited by 1,154
  12. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  13. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  14. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  15. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  16. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  17. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  18. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  19. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  20. 2019ELI5: Long Form Question Answeringcited by 567
  21. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  22. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
  23. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
  24. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  25. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  26. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  27. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
  28. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
  29. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  30. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 326
  31. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 302
  32. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  33. 2023Improving Text Embeddings with Large Language Modelscited by 295
  34. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 272
  35. 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 271
  36. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  37. 2022RealTime QA: What's the Answer Right Now?cited by 239
  38. 2023Proving Test Set Contamination in Black Box Language Modelscited by 227

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 24; 2025: 105; 2026: 71.

200 papers.

Sort:

By area:

  1. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  2. 2024LightRAG: Simple and Fast Retrieval-Augmented Generationcited by 310
  3. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  4. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAGcited by 122
  5. 2024Language agents achieve superhuman synthesis of scientific knowledgecited by 121
  6. 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 120
  7. 2025StepSearch: Igniting LLMs Search Ability via Step-Wise Proximal Policy Optimizationcited by 82
  8. 2024Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Modelscited by 57
  9. 2024ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilitiescited by 52
  10. 2024A Survey on Data Synthesis and Augmentation for Large Language Modelscited by 40
  11. 2025Parametric Retrieval Augmented Generationcited by 38
  12. 2024JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Rerankingcited by 34
38 more
  1. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directionscited by 30
  2. 2024SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domainscited by 28
  3. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontierscited by 26
  4. 2025MA-RAG: Multi-Agent Retrieval-Augmented Generation via Collaborative Chain-of-Thought Reasoningcited by 24
  5. 2025Hyper-RAG: Combating LLM Hallucinations using Hypergraph-Driven Retrieval-Augmented Generationcited by 23
  6. 2024RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignmentcited by 22
  7. 2025O2-Searcher: A Searching-based Agent Model for Open-Domain Open-Ended Question Answeringcited by 22
  8. 2025IterResearch: Rethinking Long-Horizon Agents with Interaction Scalingcited by 22
  9. 2024Augmenting Multimodal LLMs with Self-Reflective Tokens for Knowledge-based Visual Question Answeringcited by 20
  10. 2025U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystackcited by 20
  11. 2025DualRAG: A Dual-Process Approach to Integrate Reasoning and Retrieval for Multi-Hop Question Answeringcited by 20
  12. 2024An overview of domain-specific foundation model: key technologies, applications and challengescited by 19
  13. 2024SFR-RAG: Towards Contextually Faithful LLMscited by 19
  14. 2025Collab-RAG: Boosting Retrieval-Augmented Generation for Complex Question Answering via White-Box and Black-Box LLM Collaborationcited by 19
  15. 2025Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-Tuningcited by 17
  16. 2025BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actionscited by 17
  17. 2025FinRAGBench-V: A Benchmark for Multimodal RAG with Visual Citation in the Financial Domaincited by 16
  18. 2025Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compressioncited by 15
  19. 2025RAG-R1: Incentivizing the Search and Reasoning Capabilities of LLMs through Multi-query Parallelismcited by 15
  20. 2024OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMscited by 14
  21. 2025DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generationcited by 14
  22. 2025Retrieval-augmented GUI Agents with Generative Guidelinescited by 14
  23. 2025RaCoT: Plug-and-Play Contrastive Example Generation Mechanism for Enhanced LLM Reasoning Reliabilitycited by 14
  24. 2025Training a Utility-based Retriever Through Shared Context Attribution for Retrieval-Augmented Language Modelscited by 13
  25. 2025Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metricscited by 12
  26. 2025ARAG: Agentic Retrieval Augmented Generation for Personalized Recommendationcited by 12
  27. 2024PA-RAG: RAG Alignment via Multi-Perspective Preference Optimizationcited by 11
  28. 2025Iterative Self-Incentivization Empowers Large Language Models as Agentic Searcherscited by 11
  29. 2025UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversationscited by 11
  30. 2026OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAGcited by 11
  31. 2026Coding Agents are Effective Long-Context Processorscited by 11
  32. 2024RadioRAG: Online Retrieval-augmented Generation for Radiology Question Answeringcited by 10
  33. 2025LevelRAG: Enhancing Retrieval-Augmented Generation with Multi-hop Logic Planning over Rewriting Augmented Searcherscited by 10
  34. 2025C-3PO: Compact Plug-and-Play Proxy Optimization to Achieve Human-like Retrieval-Augmented Generationcited by 9
  35. 2025RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimizationcited by 9
  36. 2025CMCTS: A Constrained Monte Carlo Tree Search Framework for Mathematical Reasoning in Large Language Modelcited by 9
  37. 2025Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Modelscited by 9
  38. 2025MobileRAG: A Fast, Memory-Efficient, and Energy-Efficient Method for On-Device RAGcited by 9

50 most cited of 200 All 200 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Tuning LLMs by RAG Principles: Towards LLM-native Memory
  2. 2025Post-training an LLM for RAG? Train on Self-Generated Demonstrations
  3. 2024In Defense of RAG in the Era of Long-Context Language Models
  4. 2024RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generation
  5. 2025PRGB Benchmark: A Robust Placeholder-Assisted Algorithm for Benchmarking Retrieval-Augmented Generation
  6. 2025VaccineRAG: Boosting Multimodal Large Language Models' Immunity to Harmful RAG Samples
  7. 2024Bridging the Preference Gap between Retrievers and LLMs
  8. 2024LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering