mlsanityData to 14 Oct 2026

RAG-Instruct: Boosting LLMs with Diverse Retrieval-Augmented Instructions

arXiv 2501.00353 ↗First posted 31 Dec 2024cs.CL cs.AI cs.LGcited by 25

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) has emerged as a key paradigm for enhancing large language models (LLMs) by incorporating external knowledge. However, current RAG methods face two limitations: (1) they only cover limited RAG scenarios. (2) They suffer from limited task diversity due to the lack of a general RAG dataset. To address these limitations, we propose RAG-Instruct, a general method for synthesizing diverse and high-quality RAG instruction data based on any source corpus. Our approach leverages (1) five RAG paradigms, which encompass diverse query-document relationships, and (2) instruction simulation, which enhances instruction diversity and quality by utilizing the strengths of existing instruction datasets. Using this method, we construct a 40K instruction dataset from Wikipedia, comprehensively covering diverse RAG scenarios and tasks. Experiments demonstrate that RAG-Instruct effectively enhances LLMs' RAG capabilities, achieving strong zero-shot performance and significantly outperforming various RAG baselines across a diverse set of tasks. RAG-Instruct is publicly available at https://github.com/FreedomIntelligence/RAG-Instruct.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 18 of its 56 references that are in the map. First posted on arXiv 31 Dec 2024.

Builds on

18 of 56 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research22 May

Other references · 17

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2024Qwen2 Technical Reportcited by 2,120
  6. 2023Active Retrieval Augmented Generationcited by 516
  7. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 461
  8. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 262
  9. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 195
  10. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 159
  11. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 147
  12. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 101
5 more
  1. 2023HuatuoGPT-II, One-stage Training for Medical Adaption of LLMscited by 92
  2. 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAGcited by 90
  3. 2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
  4. 2023Interpretable Unified Language Checkingcited by 26
  5. 2024A Compressive Memory-based Retrieval Approach for Event Argument Extractioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 19; 2026: 6.

25 papers.

Sort:

By area:

  1. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 166
  2. 2025HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrievalcited by 32
  3. 2025How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmarkcited by 28
  4. 2025AttentionRAG: Attention-Guided Context Pruning in Retrieval-Augmented Generationcited by 19
  5. 2025Unmasking Digital Falsehoods: A Comparative Analysis of LLM-Based Misinformation Detection Strategiescited by 16
  6. 2025AMAS: Adaptively Determining Communication Topology for LLM-based Multi-Agent Systemcited by 15
  7. 2025LastingBench: Defend Benchmarks Against Knowledge Leakagecited by 14
  8. 2025RaCoT: Plug-and-Play Contrastive Example Generation Mechanism for Enhanced LLM Reasoning Reliabilitycited by 14
  9. 2025UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter-Efficient Fine-Tuning of Large Modelscited by 11
  10. 2025SelfAug: Mitigating Catastrophic Forgetting in Retrieval-Augmented Generation via Distribution Self-Alignmentcited by 9
  11. 2025Enhancing Customer Contact Efficiency with Graph Neural Networks in Credit Card Fraud Detection Workflowcited by 8
  12. 2025FT-MDT: Extracting Decision Trees from Medical Texts via a Novel Low-rank Adaptation Methodcited by 7
13 more
  1. 2025Image-to-Image Translation with Diffusion Transformers and CLIP-Based Image Conditioningcited by 6
  2. 2025Mixture of Length and Pruning Experts for Knowledge Graphs Reasoningcited by 5
  3. 2025A Survey of Link Prediction in N-ary Knowledge Graphscited by 4
  4. 2026End-to-End Context Compression at Scalecited by 4
  5. 2025A Hybrid Transformer Model for Fake News Detection: Leveraging Bayesian Optimization and Bidirectional Recurrent Unitcited by 3
  6. 2025Coarse-to-Fine Personalized LLM Impressions for Streamlined Radiology Reportscited by 1
  7. 2025Beyond Function-Level Search: Repository-Aware Dual-Encoder Code Retrieval with Adversarial Verificationcited by 1
  8. 2025ToM: Leveraging Tree-oriented MapReduce for Long-Context Reasoning in Large Language Modelscited by 1
  9. 2026DIVERGE: Diversity-Enhanced RAG for Open-Ended Information Seekingcited by 1
  10. 2026How Fine-Grained Should a RAG Benchmark Be? A Hierarchical Framework for Synthetic Question Generationcited by 1
  11. 2026From Experts to Sub-experts: Fine-grained Parameter-Efficient Fine-Tuning for MoE LLMscited by 1
  12. 2026Injecting Knowledge from Social Science Journals to Improve Indonesian Cultural Understanding by LLMscited by 0
  13. 2026Bridging Auxiliary Constraints to Resolve Instruction Following in Large Reasoning Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systems
  2. 2024RAG Foundry: A Framework for Enhancing LLMs for Retrieval Augmented Generation
  3. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
  4. 2025DACL-RAG: Data Augmentation Strategy with Curriculum Learning for Retrieval-Augmented Generation
  5. 2024Toward General Instruction-Following Alignment for Retrieval-Augmented Generation
  6. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting
  7. 2024Towards Understanding Systems Trade-offs in Retrieval-Augmented Generation Model Inference
  8. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models