mlsanityData to 7 Oct 2026

Integrating Physician Diagnostic Logic into Large Language Models: Preference Learning from Process Feedback

arXiv 2401.05695 ↗First posted 11 Jan 2024cs.CLcited by 9

On the map: LLM alignment / RLHF / preference optimisation → Medical LLM alignment

Abstract (arXiv)

The use of large language models in medical dialogue generation has garnered significant attention, with a focus on improving response quality and fluency. While previous studies have made progress in optimizing model performance for single-round medical Q&A tasks, there is a need to enhance the model's capability for multi-round conversations to avoid logical inconsistencies. To address this, we propose an approach called preference learning from process feedback~(PLPF), which integrates the doctor's diagnostic logic into LLMs. PLPF involves rule modeling, preference data generation, and preference alignment to train the model to adhere to the diagnostic process. Experimental results using Standardized Patient Testing show that PLPF enhances the diagnostic accuracy of the baseline model in medical conversations by 17.6%, outperforming traditional reinforcement learning from human feedback. Additionally, PLPF demonstrates effectiveness in both multi-round and single-round dialogue tasks, showcasing its potential for improving medical dialogue generation.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Medical LLM alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 18 of its 38 references that are in the map. First posted on arXiv 11 Jan 2024.

Builds on

18 of 38 references.

References · 18

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2023Qwen Technical Reportcited by 3,273
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  5. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  6. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  7. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  8. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  9. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
  10. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
  11. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  12. 2023DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Taskcited by 130
6 more
  1. 2023HuatuoGPT-II, One-stage Training for Medical Adaption of LLMscited by 91
  2. 2023DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultationcited by 89
  3. 2023ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluationcited by 85
  4. 2020MedDG: An Entity-Centric Medical Consultation Dataset for Entity-Aware Medical Dialogue Generationcited by 45
  5. 2022A Benchmark for Automatic Medical Consultation System: Frameworks, Tasks and Datasetscited by 45
  6. 2023PlugMed: Improving Specificity in Patient-Centered Medical Dialogue Generation using In-Context Learningcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 4.

9 papers.

  1. 2025MedGUIDE: Benchmarking Clinical Decision-Making in Large Language Modelscited by 10
  2. 2026Baichuan-M3: Modeling Clinical Inquiry for Reliable Medical Decision-Makingcited by 10
  3. 2024Direct Preference Optimization for Suppressing Hallucinated Prior Exams in Radiology Report Generationcited by 7
  4. 2024RuleAlign: Making Large Language Models Better Physicians with Diagnostic Rule Alignmentcited by 3
  5. 2024Exploring LLM-based Data Annotation Strategies for Medical Dialogue Preference Alignmentcited by 1
  6. 2026From Triage to Discharge: A Survey of NLP Tasks, Methods, and Open Challenges in the Emergency Departmentcited by 1
  7. 2025CMQCIC-Bench: A Chinese Benchmark for Evaluating Large Language Models in Medical Quality Control Indicator Calculationcited by 0
  8. 2026Med-Evo: Test-time Self-evolution for Medical Multimodal Large Language Modelscited by 0
  9. 2026CAREAgent: Clinical Agent with Structured Reasoning and Tool-Integrated for Order Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Reasoning Like a Doctor: Improving Medical Dialogue Systems via Diagnostic Reasoning Process Alignment
  2. 2025IIMedGPT: Promoting Large Language Model Capabilities of Medical Tasks by Efficient Human Preference Alignment
  3. 2025Dialogue is Better Than Monologue: Instructing Medical LLMs via Strategical Conversations
  4. 2025Conversation Forests: The Key to Fine Tuning Large Language Models for Multi-Turn Medical Conversations is Branching
  5. 2025Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications
  6. 2026MedDialogRubrics: A Comprehensive Benchmark and Evaluation Framework for Multi-turn Medical Consultations in Large Language Models
  7. 2023Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialogue
  8. 2026PCQC: Privileged Counterfactual Question Credit for Multi-Turn Medical Dialogue