mlsanityData to 7 Oct 2026

A Comparison of Independent and Joint Fine-tuning Strategies for Retrieval-Augmented Generation

arXiv 2510.01600 ↗First posted 2 Oct 2025cs.CL cs.AIcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

A Comparison of Independent and Joint Fine-tuning Strategies for Retrieval-Augmented Generation Download PDF Neal Gregory Lawton, Alfy Samuel, Anoop Kumar, Daben Liu Published: 20 Aug 2025, Retrieval augmented generation (RAG) is a popular framework for question answering that is powered by two large language models (LLMs): an embedding model that retrieves context documents from a database that are relevant to a given question, and a generator model that uses the retrieved context to generate an answer to the question. Both the embedding and generator models can be fine-tuned to increase performance of a RAG pipeline on a new task, but multiple fine-tuning strategies exist with different costs and benefits. In this paper, we evaluate and compare several RAG fine-tuning strategies, including independent, joint, and two-phase fine-tuning. In our experiments, we observe that all of these strategies achieve about equal improvement in EM and F1 generation quality metrics, although they have significantly different computational costs. We conclude the optimal fine-tuning strategy to use depends on whether the training dataset includes context labels and whether a grid search over the learning rates for the embedding and generator models is required.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 11 of its 14 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

11 of 14 references.

References · 11

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  5. 2017Billion-scale similarity search with GPUscited by 2,477
  6. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  7. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 326
  8. 2017Efficient Natural Language Response Suggestion for Smart Replycited by 228
  9. 2021Scaling Deep Contrastive Learning Batch Size under Memory Limited Setupcited by 123
  10. 2019One Epoch Is All You Needcited by 41
  11. 2023Is One Epoch All You Need For Multi-Fidelity Hyperparameter Optimization?cited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024REFINE on Scarce Data: Retrieval Enhancement through Fine-Tuning via Model Fusion of Embedding Models
  2. 2026Assessment of RAG and Fine-Tuning for Industrial Question-Answering-Applications
  3. 2024Context Awareness Gate For Retrieval Augmented Generation
  4. 2024Better RAG using Relevant Information Gain
  5. 2024Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge
  6. 2025Adapting Large Language Models for Multi-Domain Retrieval-Augmented-Generation
  7. 2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions
  8. 2024Enhancing Large Language Model Performance To Answer Questions and Extract Information More Accurately