mlsanityData to 14 Oct 2026

Med42 -- Evaluating Fine-Tuning Strategies for Medical LLMs: Full-Parameter vs. Parameter-Efficient Approaches

arXiv 2404.14779 ↗First posted 23 Apr 2024cs.CLcited by 52

Not on the map. Why?

Abstract (arXiv)

This study presents a comprehensive analysis and comparison of two predominant fine-tuning methodologies - full-parameter fine-tuning and parameter-efficient tuning - within the context of medical Large Language Models (LLMs). We developed and refined a series of LLMs, based on the Llama-2 architecture, specifically designed to enhance medical knowledge retrieval, reasoning, and question-answering capabilities. Our experiments systematically evaluate the effectiveness of these tuning strategies across various well-known medical benchmarks. Notably, our medical LLM Med42 showed an accuracy level of 72% on the US Medical Licensing Examination (USMLE) datasets, setting a new standard in performance for openly available medical LLMs. Through this comparative analysis, we aim to identify the most effective and efficient method for fine-tuning LLMs in the medical domain, thereby contributing significantly to the advancement of AI-driven healthcare applications.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). Builds on 21 of its 36 references that are in the map. First posted on arXiv 23 Apr 2024.

Builds on

21 of 36 references.

References · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  6. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  7. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  8. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  9. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  10. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  11. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  12. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
9 more
  1. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 286
  2. 2023A Study of Generative Large Language Model for Medical Research and Healthcarecited by 163
  3. 2020Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learningcited by 139
  4. 2019A Question-Entailment Approach to Question Answeringcited by 134
  5. 2023A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasetscited by 123
  6. 2019HEAD-QA: A Healthcare Dataset for Complex Reasoningcited by 101
  7. 2023PMC-LLaMA: Towards Building Open-source Language Models for Medicinecited by 88
  8. 2023Clinical Camel: An Open Expert-Level Medical Language Model with Dialogue-Based Knowledge Encodingcited by 71
  9. 2023Parameter-Efficient Fine-Tuning without Introducing New Latencycited by 59

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 16; 2025: 26; 2026: 10.

52 papers.

Sort:

By area:

  1. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 265
  2. 2024UltraMedical: Building Specialized Generalists in Biomedicinecited by 67
  3. 2024Low-Rank Adaptation for Foundation Models: A Comprehensive Reviewcited by 37
  4. 2025Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with Minimalist Rule-Based RLcited by 36
  5. 2024LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Modelscited by 32
  6. 2024Medical Adaptation of Large Language and Vision-Language Models: Are We Making Progress?cited by 28
  7. 2024MEDIC: Comprehensive Evaluation of Leading Indicators for LLM Safety and Utility in Clinical Applicationscited by 25
  8. 2025Automating Expert-Level Medical Reasoning Evaluation of Large Language Modelscited by 22
  9. 2024Biomedical Large Languages Models Seem not to be Superior to Generalist Models on Unseen Medical Datacited by 21
  10. 2024BiMediX2: Bio-Medical EXpert LMM for Diverse Medical Modalitiescited by 20
  11. 2024Do Large Language Models Align with Core Mental Health Counseling Competencies?cited by 19
  12. 2025Citrus: Leveraging Expert Cognitive Pathways in a Medical Language Model for Advanced Medical Decision Supportcited by 16
38 more
  1. 2025TabPFN-Wide: Continued Pre-Training for Extreme Feature Countscited by 16
  2. 2024Do Language Models Understand Time?cited by 14
  3. 2025ORANSight-2.0: Foundational LLMs for O-RANcited by 13
  4. 2024Automating Exploratory Proteomics Research via Language Modelscited by 11
  5. 2024Towards Adapting Open-Source Large Language Models for Expert-Level Clinical Note Generationcited by 9
  6. 2024The Limited Impact of Medical Adaptation of Large Language and Vision-Language Modelscited by 9
  7. 2025Uncertainty-Aware Large Language Models for Explainable Disease Diagnosiscited by 9
  8. 2024Beyond Fine-tuning: Unleashing the Potential of Continuous Pretraining for Clinical LLMscited by 8
  9. 2026Conformal Selective Acting: Anytime-Valid Risk Control for RLVR-Trained LLMscited by 8
  10. 2025ShizhenGPT: Towards Multimodal LLMs for Traditional Chinese Medicinecited by 7
  11. 2025Prompt to Polyp: Medical Text-Conditioned Image Synthesis with Diffusion Modelscited by 5
  12. 2024All-in-One Tuning and Structural Pruning for Domain-Specific LLMscited by 4
  13. 2025Enhancing the Medical Context-Awareness Ability of LLMs via Multifaceted Self-Refinement Learningcited by 4
  14. 2025MediEval: A Unified Medical Benchmark for Patient-Contextual and Knowledge-Grounded Reasoning in LLMscited by 4
  15. 2025ConTextual: Improving Clinical Text Summarization in LLMs with Context-preserving Token Filtering and Knowledge Graphscited by 3
  16. 2025Evaluating LLMs Across Multi-Cognitive Levels: From Medical Knowledge Mastery to Scenario-Based Problem Solvingcited by 3
  17. 2026SoK: Privacy-aware LLM in Healthcare: Threat Model, Privacy Techniques, Challenges and Recommendationscited by 3
  18. 2024The Potential of LLMs in Medical Education: Generating Questions and Answers for Qualification Examscited by 2
  19. 2025Collaboration among Multiple Large Language Models for Medical Question Answeringcited by 2
  20. 2025AdaThink-Med: Optimizing Inference-Time Compute for Medical Reasoning via Uncertainty Quantificationcited by 2
  21. 2026A decodability criterion predicts when hidden-state selection beats majority voting in large language modelscited by 2
  22. 2025Pastiche Novel Generation Creating: Fan Fiction You Love in Your Favorite Author's Stylecited by 1
  23. 2025Enabling Flexible Multi-LLM Integration for Scalable Knowledge Aggregationcited by 1
  24. 2025Achieving Trustworthy Real-Time Decision Support Systems with Low-Latency Interpretable AI Modelscited by 1
  25. 2025CrunchLLM: Multitask LLMs for Structured Business Reasoning and Outcome Predictioncited by 1
  26. 2025Reliability of Large Language Model Generated Clinical Reasoning in Assisted Reproductive Technology: Blinded Comparative Evaluation Studycited by 1
  27. 2025Knowledge Elicitation with Large Language Models for Interpretable Cancer Stage Identification from Pathology Reportscited by 1
  28. 2025Boosting In-Silicon Directed Evolution with Fine-Tuned Protein Language Model and Tree Searchcited by 1
  29. 2026MedPriv-Bench: Benchmarking the Privacy-Utility Trade-off of Large Language Models in Medical Open-Ended Question Answeringcited by 1
  30. 2026Benchmarking the Safety of Large Language Models for Robotic Health Attendant Controlcited by 1
  31. 2025PrinciplismQA: A Philosophy-Grounded Approach to Assessing LLM-Human Clinical Medical Ethics Alignmentcited by 0
  32. 2025Generative Foundation Model for Structured and Unstructured Electronic Health Recordscited by 0
  33. 2025Scam Shield: Multi-Model Voting and Fine-Tuned LLMs Against Adversarial Attackscited by 0
  34. 2025ChEmREF: Evaluating Language Model Readiness for Chemical Emergency Responsecited by 0
  35. 2025Hierarchical Dual-Strategy Unlearning for Biomedical and Healthcare Intelligence Using Imperfect and Privacy-Sensitive Medical Datacited by 0
  36. 2026Fine-Tune, Don't Prompt, Your Language Model to Identify Biased Language in Clinical Notescited by 0
  37. 2026EpiScreen: Early Epilepsy Detection from Electronic Health Records with Large Language Modelscited by 0
  38. 2026Towards the Next Frontier of LLMs, Training on Private Data: A Cross-Domain Benchmark for Federated Fine-Tuningcited by 0

50 most cited of 52 All 52 (JSON)