mlsanityData to 7 Oct 2026

Fine Tuning Large Language Models for Medicine: The Role and Importance of Direct Preference Optimization

arXiv 2409.12741 ↗First posted 19 Sep 2024cs.CL cs.AIcited by 5

On the map: LLM alignment / RLHF / preference optimisation → Medical LLM alignment

Abstract (arXiv)

Large Language Model (LLM) fine tuning is underutilized in the field of medicine. Two of the most common methods of fine tuning are Supervised Fine Tuning (SFT) and Direct Preference Optimization (DPO), but there is little guidance informing users when to use either technique. In this investigation, we compare the performance of SFT and DPO for five common natural language tasks in medicine: Classification with text data, Classification with numeric data, Clinical Reasoning, Summarization, and Clinical Triage. We find that SFT alone is sufficient for Classification with text data, whereas DPO improves performance for the more complex tasks of Clinical Reasoning, Summarization and Clinical Triage. Our results establish the role and importance of DPO fine tuning within medicine, and consequently call attention to current software gaps that prevent widespread deployment of this technique.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Medical LLM alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 7 of its 18 references that are in the map. First posted on arXiv 19 Sep 2024.

Builds on

7 of 18 references.

References · 7

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 272
  3. 2023Large language models in healthcare and medical domain: A reviewcited by 126
  4. 2023Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicinecited by 83
  5. 2024Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspectivecited by 69
  6. 2023Large Language Models to Identify Social Determinants of Health in Electronic Health Recordscited by 55
  7. 2024Insights into Alignment: Evaluating DPO and its Variants Across Multiple Taskscited by 35

Built on by

Cited by year

Citing papers per year: 2025: 5.

5 papers.

  1. 2025The Rise of Small Language Models in Healthcare: A Comprehensive Surveycited by 18
  2. 2025Eeyore: Realistic Depression Simulation via Supervised and Preference Optimizationcited by 3
  3. 2025Conversation Forests: The Key to Fine Tuning Large Language Models for Multi-Turn Medical Conversations is Branchingcited by 2
  4. 2025Deep Learning-based Prediction of Clinical Trial Enrollment with Uncertainty Estimatescited by 0
  5. 2025FocusMed: A Large Language Model-based Framework for Enhancing Medical Question Summarization with Focus Identificationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Model
  2. 2024Biomedical Large Languages Models Seem not to be Superior to Generalist Models on Unseen Medical Data
  3. 2025IIMedGPT: Promoting Large Language Model Capabilities of Medical Tasks by Efficient Human Preference Alignment
  4. 2025FineMedLM-o1: Enhancing Medical Knowledge Reasoning Ability of LLM from Supervised Fine-Tuning to Test-Time Training
  5. 2023ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences
  6. 2024Can LLMs' Tuning Methods Work in Medical Multimodal Domain?
  7. 2025Improving LLM Safety and Helpfulness using SFT and DPO: A Study on OPT-350M
  8. 2025Generalization of Medical Large Language Models through Cross-Domain Weak Supervision