mlsanityData to 14 Oct 2026

Fine-Tuning LLMs for Reliable Medical Question-Answering Services

arXiv 2410.16088 ↗First posted 21 Oct 2024cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

We present an advanced approach to medical question-answering (QA) services, using fine-tuned Large Language Models (LLMs) to improve the accuracy and reliability of healthcare information. Our study focuses on optimizing models like LLaMA-2 and Mistral, which have shown great promise in delivering precise, reliable medical answers. By leveraging comprehensive datasets, we applied fine-tuning techniques such as rsDoRA+ and ReRAG. rsDoRA+ enhances model performance through a combination of decomposed model weights, varied learning rates for low-rank matrices, and rank stabilization, leading to improved efficiency. ReRAG, which integrates retrieval on demand and question rewriting, further refines the accuracy of the responses. This approach enables healthcare providers to access fast, dependable information, aiding in more efficient decision-making and fostering greater patient trust. Our work highlights the potential of fine-tuned LLMs to significantly improve the quality and accessibility of medical information services, ultimately contributing to better healthcare outcomes for all.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 17 of its 24 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

17 of 24 references.

References · 17

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  4. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  5. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 758
  6. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  7. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 656
  8. 2024LoRA+: Efficient Low Rank Adaptation of Large Modelscited by 377
  9. 2023AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuningcited by 357
  10. 2023A Rank Stabilization Scaling Factor for Fine-Tuning with LoRAcited by 243
  11. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 223
  12. 2023When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applicationscited by 111
5 more
  1. 2024Data-efficient Fine-tuning for LLM-based Recommendationcited by 104
  2. 2023Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-Incited by 71
  3. 2024Enhancing Large Language Model Performance To Answer Questions and Extract Information More Accuratelycited by 21
  4. 2024Large Language Models and User Trust: Consequence of Self-Referential Learning Loop and the Deskilling of Healthcare Professionalscited by 13
  5. 2023Powering an AI Chatbot with Expert Sourcing to Support Credible Health Information Accesscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Trustworthy Medical Question Answering: An Evaluation-Centric Surveycited by 7
  2. 2026SoK: Privacy-aware LLM in Healthcare: Threat Model, Privacy Techniques, Challenges and Recommendationscited by 3
  3. 2025Detecting and Understanding Hateful Contents in Memes Through Captioning and Visual Question-Answeringcited by 2
  4. 2025Simplified Swarm Learning Framework for Robust and Scalable Diagnostic Services in Cancer Histopathologycited by 2