# Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge

arXiv: https://arxiv.org/abs/2403.01432 · HTML: https://mlsanity.com/paper/2403.01432/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 47 of its 66 references that are in the map.
- First posted on arXiv 3 Mar 2024.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 30 of 47)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models](https://mlsanity.com/paper/2106.10199/) (2021)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning](https://mlsanity.com/paper/2205.05638/) (2022)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [TinyLlama: An Open-Source Small Language Model](https://mlsanity.com/paper/2401.02385/) (2024)
- [Zephyr: Direct Distillation of LM Alignment](https://mlsanity.com/paper/2310.16944/) (2023)
- [Large Language Models Struggle to Learn Long-Tail Knowledge](https://mlsanity.com/paper/2211.08411/) (2022)
- [The Power of Noise: Redefining Retrieval for RAG Systems](https://mlsanity.com/paper/2401.14887/) (2024)
- [Fine-Tuning LLaMA for Multi-Stage Text Retrieval](https://mlsanity.com/paper/2310.08319/) (2023)
- [Pretrained Transformers for Text Ranking: BERT and Beyond](https://mlsanity.com/paper/2010.06467/) (2020)
- [Challenges and Applications of Large Language Models](https://mlsanity.com/paper/2307.10169/) (2023)
- [RealTime QA: What's the Answer Right Now?](https://mlsanity.com/paper/2207.13332/) (2022)
- [PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them](https://mlsanity.com/paper/2102.07033/) (2021)
- [Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs](https://mlsanity.com/paper/2312.05934/) (2023)
- [Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?](https://mlsanity.com/paper/2308.10168/) (2023)
- [Synthetic QA Corpora Generation with Roundtrip Consistency](https://mlsanity.com/paper/1906.05416/) (2019)
- [Simple Entity-Centric Questions Challenge Dense Retrievers](https://mlsanity.com/paper/2109.08535/) (2021)

## Built on by (most cited first, 30 of 50)

- [PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel Optimization](https://mlsanity.com/paper/2504.07717/) (2025)
- [How Much Can RAG Help the Reasoning of LLM?](https://mlsanity.com/paper/2410.02338/) (2024)
- [A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions](https://mlsanity.com/paper/2507.18910/) (2025)
- [A Survey of Personalization: From RAG to Agent](https://mlsanity.com/paper/2504.10147/) (2025)
- [A Survey on Recent Advances in Conversational Data Generation](https://mlsanity.com/paper/2405.13003/) (2024)
- [MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledge](https://mlsanity.com/paper/2412.17032/) (2024)
- [Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compression](https://mlsanity.com/paper/2505.19433/) (2025)
- [Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models](https://mlsanity.com/paper/2501.05382/) (2025)
- [RAGVA: Engineering Retrieval Augmented Generation-based Virtual Assistants in Practice](https://mlsanity.com/paper/2502.14930/) (2025)
- [The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM Compression Preserve?](https://mlsanity.com/paper/2502.17535/) (2025)
- [Uncertainty Quantification for Retrieval-Augmented Reasoning](https://mlsanity.com/paper/2510.11483/) (2025)
- [GPT vs RETRO: Exploring the Intersection of Retrieval and Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2407.04528/) (2024)
- [Rote Learning Considered Useful: Generalizing over Memorized Data in LLMs](https://mlsanity.com/paper/2507.21914/) (2025)
- [Total Recall QA: A Verifiable Evaluation Suite for Deep Research Agents](https://mlsanity.com/paper/2603.18516/) (2026)
- [Optimizing Retrieval Augmented Generation for Object Constraint Language](https://mlsanity.com/paper/2505.13129/) (2025)
- [A Comprehensive Taxonomy of Negation for NLP and Neural Retrievers](https://mlsanity.com/paper/2507.22337/) (2025)
- [Prompting Large Language Models with Partial Knowledge for Answering Questions with Unseen Entities](https://mlsanity.com/paper/2508.01290/) (2025)
- [RAGTrace: Understanding and Refining Retrieval-Generation Dynamics in Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.06056/) (2025)
- [Creating a Taxonomy for Retrieval Augmented Generation Applications](https://mlsanity.com/paper/2408.02854/) (2024)
- [KBAlign: Efficient Self Adaptation on Specific Knowledge Bases](https://mlsanity.com/paper/2411.14790/) (2024)
- [Can We Edit LLMs for Long-Tail Biomedical Knowledge?](https://mlsanity.com/paper/2504.10421/) (2025)
- [Grow Up and Merge: Scaling Strategies for Efficient Language Adaptation](https://mlsanity.com/paper/2512.10772/) (2025)
- [Transforming External Knowledge into Triplets for Enhanced Retrieval in RAG of LLMs](https://mlsanity.com/paper/2604.12610/) (2026)
- [When Deep Research Agents Stagnate: Enhancing Reasoning with Retrieval-Aware Agent Control](https://mlsanity.com/paper/2608.15191/) (2026)
- [QMOS: Enhancing LLMs for Telecommunication with Question Masked loss and Option Shuffling](https://mlsanity.com/paper/2409.14175/) (2024)
- [Quantifying Memorization and Parametric Response Rates in Retrieval-Augmented Vision-Language Models](https://mlsanity.com/paper/2502.13836/) (2025)
- [Document Retrieval Augmented Fine-Tuning (DRAFT) for safety-critical software assessments](https://mlsanity.com/paper/2505.01307/) (2025)
- [Diffusion-Inspired Masked Fine-Tuning for Knowledge Injection in Autoregressive LLMs](https://mlsanity.com/paper/2510.09885/) (2025)
- [Uncovering inequalities in new knowledge learning by large language models across different languages](https://mlsanity.com/paper/2503.04064/) (2025)
- [Optimization of embeddings storage for RAG systems using quantization and dimensionality reduction techniques](https://mlsanity.com/paper/2505.00105/) (2025)

## Abstract (arXiv)

Language Models (LMs) memorize a vast amount of factual knowledge, exhibiting strong performance across diverse tasks and domains. However, it has been observed that the performance diminishes when dealing with less-popular or low-frequency concepts and entities, for example in domain specific applications. The two prominent approaches to enhance the performance of LMs on low-frequent topics are: Retrieval Augmented Generation (RAG) and fine-tuning (FT) over synthetic data. This paper explores and evaluates the impact of RAG and FT on customizing LMs in handling low-frequency entities on question answering tasks. We conduct extensive experiments on twelve LMs of varying size and type and different fine tuning, data augmentation, and retrieval models. Our findings indicate that while FT boosts the performance across entities of varying popularity, RAG surpasses FT by a large margin particularly for least popular factual knowledge. Additionally, the success of both RAG and FT approaches is amplified by improving retrieval and data augmentation techniques. Fine tuning, while beneficial for small LMs, requires extensive resources. To address this issue, we propose the new Stimulus RAG approach that surpasses the effectiveness of fine tuning based approaches, thereby eliminating the need for the costly data augmentation and fine tuning step for enriching LMs with less popular factual knowledge. The code is available at urlhttps://github.com/informagi/RAGvsFT.