Domain Fine-Tuning vs. Retrieval-Augmented Generation for Medical Multiple-Choice Question Answering: A Controlled Comparison at the 4B-Parameter Scale
On the map: Retrieval-augmented generation → Medical and biomedical RAG
Abstract (arXiv)
Practitioners deploying small open-weight large language models (LLMs) for medical question answering face a recurring design choice: invest in a domain-fine-tuned model, or keep a general-purpose model and inject domain knowledge at inference time via retrieval-augmented generation (RAG). We isolate this trade-off by holding model size, prompt template, decoding temperature, retrieval pipeline, and evaluation protocol fixed, and varying only (i) whether the model has been domain-adapted (Gemma 3 4B vs. MedGemma 4B, both 4-bit quantized and served via Ollama) and (ii) whether retrieved passages from a medical knowledge corpus are inserted into the prompt. We evaluate all four cells of this 2x2 design on the full MedQA-USMLE 4-option test split (1,273 questions) with three repetitions per question (15,276 LLM calls). Domain fine-tuning yields a +6.8 percentage-point gain in majority-vote accuracy over the general 4B baseline (53.3% vs. 46.4%, McNemar p < 10^-4). RAG over MedMCQA explanations does not produce a statistically significant gain in either model, and in the domain-tuned model the point estimate is slightly negative (-1.9 pp, p = 0.16). At this scale and on this benchmark, domain knowledge encoded in weights dominates domain knowledge supplied in context. We release the full experiment code and JSONL traces to support replication.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical and biomedical RAG, one of 15 directions in Retrieval-augmented generation. Builds on 10 of its 13 references that are in the map. First posted on arXiv 26 Apr 2026.
Builds on
10 of 13 references.
Closest ancestors · 1
Same direction, newest first
Other references · 9
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022Large Language Models Encode Clinical Knowledgecited by 1,807
- 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
- 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
- 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
- 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
- 2025MedGemma Technical Reportcited by 398
- 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 255
- 2023Augmenting Black-box LLMs with Medical Textbooks for Biomedical Question Answeringcited by 11
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Optimizing Medical Question-Answering Systems: A Comparative Study of Fine-Tuned and Zero-Shot Large Language Models with RAG Framework
- 2024Comprehensive and Practical Evaluation of Retrieval-Augmented Generation Systems for Medical Question Answering
- 2023MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answering
- 2024The Limited Impact of Medical Adaptation of Large Language and Vision-Language Models
- 2026A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering
- 2026To Adapt or not to Adapt, Rethinking the Value of Medical Knowledge-Aware Large Language Models
- 2026Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering
- 2025MedBioLM: Optimizing Medical and Biological QA with Fine-Tuned Large Language Models and Retrieval-Augmented Generation