mlsanityData to 14 Oct 2026

Federated Learning and RAG Integration: A Scalable Approach for Medical Large Language Models

arXiv 2412.13720 ↗First posted 18 Dec 2024cs.CL cs.AIcited by 4

On the map: Retrieval-augmented generation → Medical and biomedical RAG

Abstract (arXiv)

This study analyzes the performance of domain-specific Large Language Models (LLMs) for the medical field by integrating Retrieval-Augmented Generation (RAG) systems within a federated learning framework. Leveraging the inherent advantages of federated learning, such as preserving data privacy and enabling distributed computation, this research explores the integration of RAG systems with models trained under varying client configurations to optimize performance. Experimental results demonstrate that the federated learning-based models integrated with RAG systems consistently outperform their non-integrated counterparts across all evaluation metrics. This study highlights the potential of combining federated learning and RAG systems for developing domain-specific LLMs in the medical field, providing a scalable and privacy-preserving solution for enhancing text generation capabilities.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Medical and biomedical RAG, one of 15 directions in Retrieval-augmented generation. Builds on 16 of its 21 references that are in the map. First posted on arXiv 18 Dec 2024.

Builds on

16 of 21 references.

References · 16

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  4. 2024Large Language Models: A Surveycited by 525
  5. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 455
  6. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 418
  7. 2024The Faiss librarycited by 376
  8. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 286
  9. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 185
  10. 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 149
  11. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 128
  12. 2020Optimal Client Sampling for Federated Learningcited by 124
4 more
  1. 2023FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Modelscited by 63
  2. 2023Federated Learning with Regularized Client Participationcited by 14
  3. 2022A Survey on Participant Selection for Federated Learning in Mobile Networkscited by 11
  4. 2024A Fine-tuning Enhanced RAG System with Quantized Influence Measure as AI Judgecited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Talk to Right Specialists: Iterative Routing in Multi-agent Systems for Question Answeringcited by 6
  2. 2026Supercharging Federated Intelligence Retrievalcited by 0
  3. 2026Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systemscited by 0
  4. 2026Beyond Raw Context Transfer: Representation-based Federated Retrieval-Augmented Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Federated Retrieval-Augmented Generation: A Systematic Mapping Study
  2. 2025Leveraging Language Models and RAG for Efficient Knowledge Discovery in Clinical Environments
  3. 2024Development and Testing of Retrieval Augmented Generation in Large Language Models -- A Case Study Report
  4. 2025Retrieval-Augmented Generation in Medicine: A Scoping Review of Technical Implementations, Clinical Applications, and Ethical Considerations
  5. 2024TC-RAG:Turing-Complete RAG's Case study on Medical LLM Systems
  6. 2024Bailicai: A Domain-Optimized Retrieval-Augmented Generation Framework for Medical Applications
  7. 2025Privacy-Preserving Federated Embedding Learning for Localized Retrieval-Augmented Generation
  8. 2024A MapReduce Approach to Effectively Utilize Long Context Information in Retrieval Augmented Language Models