mlsanityData to 13 Oct 2026

A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systems

arXiv 2406.14972 ↗First posted 21 Jun 2024cs.CL cs.IRcited by 3

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval Augmented Generation (RAG) represents a significant advancement in artificial intelligence combining a retrieval phase with a generative phase, with the latter typically being powered by large language models (LLMs). The current common practices in RAG involve using "instructed" LLMs, which are fine-tuned with supervised training to enhance their ability to follow instructions and are aligned with human preferences using state-of-the-art techniques. Contrary to popular belief, our study demonstrates that base models outperform their instructed counterparts in RAG tasks by 20% on average under our experimental settings. This finding challenges the prevailing assumptions about the superiority of instructed LLMs in RAG applications. Further investigations reveal a more nuanced situation, questioning fundamental aspects of RAG and suggesting the need for broader discussions on the topic; or, as Fromm would have it, "Seldom is a glance at the statistics enough to understand the meaning of the figures".

Key facts. Cited by 3 papers in the map (data through 13 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 35 of its 46 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

35 of 46 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Evaluation of Retrieval-Augmented Generation: A Survey13 May
  2. 2024The Power of Noise: Redefining Retrieval for RAG Systems26 Jan

Other references · 33

  1. 2022Training language models to follow instructions with human feedbackcited by 15,856
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,221
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,986
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,797
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,716
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,412
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,925
  9. 2017Billion-scale similarity search with GPUscited by 2,490
  10. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,815
  11. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,323
  12. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,205
21 more
  1. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,111
  2. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  3. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 829
  4. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 705
  5. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 693
  6. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
  7. 2023Enabling Large Language Models to Generate Text with Citationscited by 588
  8. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  9. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 558
  10. 2023The Falcon Series of Open Language Modelscited by 533
  11. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 455
  12. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 352
  13. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 337
  14. 2024Is DPO Superior to PPO for LLM Alignment? A Comprehensive Studycited by 262
  15. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 261
  16. 2021Hurdles to Progress in Long-form Question Answeringcited by 166
  17. 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidencecited by 112
  18. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  19. 2019A Discrete Hard EM Approach for Weakly Supervised Question Answeringcited by 78
  20. 2024UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systemscited by 38
  21. 2024FaaF: Facts as a Function for the evaluation of generated textcited by 5

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systemscited by 7
  2. 2025Improving Instruct Models for Free: A Study on Partial Adaptationcited by 2
  3. 2026Automatic or Controlled? Repetition Priming Reveals Divergent Processing in Base LLMs, Instruct LLMs, and Humanscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
  2. 2024RAG-Instruct: Boosting LLMs with Diverse Retrieval-Augmented Instructions
  3. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
  4. 2024Deploying Large Language Models With Retrieval Augmented Generation
  5. 2024Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMs
  6. 2026Is Agentic RAG worth it? An experimental comparison of RAG approaches
  7. 2024Towards Understanding Systems Trade-offs in Retrieval-Augmented Generation Model Inference
  8. 2025Post-training an LLM for RAG? Train on Self-Generated Demonstrations