mlsanityData to 7 Oct 2026

Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge

arXiv 2403.01432 ↗First posted 3 Mar 2024cs.CLcited by 49

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Language Models (LMs) memorize a vast amount of factual knowledge, exhibiting strong performance across diverse tasks and domains. However, it has been observed that the performance diminishes when dealing with less-popular or low-frequency concepts and entities, for example in domain specific applications. The two prominent approaches to enhance the performance of LMs on low-frequent topics are: Retrieval Augmented Generation (RAG) and fine-tuning (FT) over synthetic data. This paper explores and evaluates the impact of RAG and FT on customizing LMs in handling low-frequency entities on question answering tasks. We conduct extensive experiments on twelve LMs of varying size and type and different fine tuning, data augmentation, and retrieval models. Our findings indicate that while FT boosts the performance across entities of varying popularity, RAG surpasses FT by a large margin particularly for least popular factual knowledge. Additionally, the success of both RAG and FT approaches is amplified by improving retrieval and data augmentation techniques. Fine tuning, while beneficial for small LMs, requires extensive resources. To address this issue, we propose the new Stimulus RAG approach that surpasses the effectiveness of fine tuning based approaches, thereby eliminating the need for the costly data augmentation and fine tuning step for enriching LMs with less popular factual knowledge. The code is available at urlhttps://github.com/informagi/RAGvsFT.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 47 of its 66 references that are in the map. First posted on arXiv 3 Mar 2024.

Builds on

47 of 66 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024The Power of Noise: Redefining Retrieval for RAG Systems26 Jan
  2. 2023Lift Yourself Up: Retrieval-augmented Text Generation with Self Memory3 May

Other references · 45

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  9. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  10. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  11. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  12. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
33 more
  1. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  2. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  3. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  4. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  5. 2023A Comprehensive Overview of Large Language Modelscited by 639
  6. 2024TinyLlama: An Open-Source Small Language Modelcited by 604
  7. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  8. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  9. 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 271
  10. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 266
  11. 2023Challenges and Applications of Large Language Modelscited by 251
  12. 2022RealTime QA: What's the Answer Right Now?cited by 239
  13. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 198
  14. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 175
  15. 2023Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?cited by 167
  16. 2019Synthetic QA Corpora Generation with Roundtrip Consistencycited by 143
  17. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  18. 2023Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluationcited by 134
  19. 2023Guiding Large Language Models via Directional Stimulus Promptingcited by 94
  20. 2024Stable LM 2 1.6B Technical Reportcited by 86
  21. 2024RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculturecited by 80
  22. 2020REL: An Entity Linker Standing on the Shoulders of Giantscited by 61
  23. 2019Unsupervised Question Answering by Cloze Translationcited by 59
  24. 2021Evaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLPcited by 30
  25. 2022Conversational Question Answering on Heterogeneous Sourcescited by 26
  26. 2024Retrieval Helps or Hurts? A Deeper Dive into the Efficacy of Retrieval Augmentation to Language Modelscited by 19
  27. 2023An Empirical Comparison of LM-based Question and Answer Generation Methodscited by 15
  28. 2023CompMix: A Benchmark for Heterogeneous Question Answeringcited by 12
  29. 2020Better Highlighting: Creating Sub-Sentence Summary Highlightscited by 11
  30. 2022Entity-aware Transformers for Entity Searchcited by 10
  31. 2022Benchmarking Long-tail Generalization with Likelihood Splitscited by 8
  32. 2023Data Augmentation for Conversational AIcited by 3
  33. 2023MMEAD: MS MARCO Entity Annotations and Disambiguationscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 27; 2026: 14.

49 papers.

Sort:

By area:

  1. 2024How Much Can RAG Help the Reasoning of LLM?cited by 31
  2. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directionscited by 30
  3. 2025PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel Optimizationcited by 29
  4. 2025A Survey of Personalization: From RAG to Agentcited by 26
  5. 2024A Survey on Recent Advances in Conversational Data Generationcited by 23
  6. 2024MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledgecited by 20
  7. 2025Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compressioncited by 15
  8. 2025Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Modelscited by 14
  9. 2025RAGVA: Engineering Retrieval Augmented Generation-based Virtual Assistants in Practicecited by 6
  10. 2025The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM Compression Preserve?cited by 6
  11. 2025Rote Learning Considered Useful: Generalizing over Memorized Data in LLMscited by 5
  12. 2025Uncertainty Quantification for Retrieval-Augmented Reasoningcited by 5
37 more
  1. 2026Total Recall QA: A Verifiable Evaluation Suite for Deep Research Agentscited by 5
  2. 2024GPT vs RETRO: Exploring the Intersection of Retrieval and Parameter-Efficient Fine-Tuningcited by 4
  3. 2025Optimizing Retrieval Augmented Generation for Object Constraint Languagecited by 4
  4. 2025A Comprehensive Taxonomy of Negation for NLP and Neural Retrieverscited by 4
  5. 2025Prompting Large Language Models with Partial Knowledge for Answering Questions with Unseen Entitiescited by 4
  6. 2025RAGTrace: Understanding and Refining Retrieval-Generation Dynamics in Retrieval-Augmented Generationcited by 4
  7. 2024Creating a Taxonomy for Retrieval Augmented Generation Applicationscited by 3
  8. 2024KBAlign: Efficient Self Adaptation on Specific Knowledge Basescited by 3
  9. 2025Can We Edit LLMs for Long-Tail Biomedical Knowledge?cited by 3
  10. 2025Grow Up and Merge: Scaling Strategies for Efficient Language Adaptationcited by 3
  11. 2026Transforming External Knowledge into Triplets for Enhanced Retrieval in RAG of LLMscited by 3
  12. 2024QMOS: Enhancing LLMs for Telecommunication with Question Masked loss and Option Shufflingcited by 2
  13. 2025Quantifying Memorization and Parametric Response Rates in Retrieval-Augmented Vision-Language Modelscited by 2
  14. 2025Document Retrieval Augmented Fine-Tuning (DRAFT) for safety-critical software assessmentscited by 2
  15. 2025Diffusion-Inspired Masked Fine-Tuning for Knowledge Injection in Autoregressive LLMscited by 2
  16. 2025Uncovering inequalities in new knowledge learning by large language models across different languagescited by 1
  17. 2025Optimization of embeddings storage for RAG systems using quantization and dimensionality reduction techniquescited by 1
  18. 2025REx86: A Local Large Language Model for Assisting in x86 Assembly Reverse Engineeringcited by 1
  19. 2026Fine-tuning vs. In-context Learning in Large Language Models: A Formal Language Learning Perspectivecited by 1
  20. 2026When Deep Research Agents Stagnate: Enhancing Reasoning with Retrieval-Aware Agent Controlcited by 1
  21. 2024WASHtsApp -- A RAG-powered WhatsApp Chatbot for supporting rural African clean water access, sanitation and hygienecited by 0
  22. 2025PDF Retrieval Augmented Question Answeringcited by 0
  23. 2025PromptAug: Fine-grained Conflict Classification Using Data Augmentationcited by 0
  24. 2025Knowledge Augmented Finetuning Matters in both RAG and Agent Based Dialog Systemscited by 0
  25. 2025MV-RAG: Retrieval Augmented Multiview Diffusioncited by 0
  26. 2025APIDA-Chat: Structured Synthesis of API Search Dialogues to Bootstrap Conversational Agentscited by 0
  27. 2025A Longitudinal Study on Different Annotator Feedback Loops in Complex RAG Taskscited by 0
  28. 2026Injecting Knowledge from Social Science Journals to Improve Indonesian Cultural Understanding by LLMscited by 0
  29. 2026Beyond Case Law: Evaluating Structure-Aware Retrieval and Safety in Statute-Centric Legal QAcited by 0
  30. 2026ChipLingo: A Systematic Training Framework for Large Language Models in EDAcited by 0
  31. 2026Assessment of RAG and Fine-Tuning for Industrial Question-Answering-Applicationscited by 0
  32. 2026Uncertainty Quantification for Multimodal Retrieval Augmented Generationcited by 0
  33. 2026Reducing Hallucinations in Complex Question Answering using Simple Graph-based Retrieval-Augmented Generation (long version)cited by 0
  34. 2026Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuningcited by 0
  35. 2026Self-evolving Agentic Customer Support System at LinkedIncited by 0
  36. 2026EvidenT: Building Trustworthy Enterprise Assistants through Evidence Groundedness and Traceabilitycited by 0
  37. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Fine-Tuning or Fine-Failing? Debunking Performance Myths in Large Language Models
  2. 2025Tuning LLMs by RAG Principles: Towards LLM-native Memory
  3. 2025Systematic Knowledge Injection into Large Language Models via Diverse Augmentation for Domain-Specific RAG
  4. 2024Enhancing Large Language Model Performance To Answer Questions and Extract Information More Accurately
  5. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
  6. 2023Establishing Performance Baselines in Fine-Tuning, Retrieval-Augmented Generation and Soft-Prompting for Non-Specialist LLM Users
  7. 2025RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defects
  8. 2024Investigating the performance of Retrieval-Augmented Generation and fine-tuning for the development of AI-driven knowledge-based systems