mlsanityData to 13 Oct 2026

Aligning Large Language Models with Healthcare Stakeholders: A Pathway to Trustworthy AI Integration

arXiv 2505.02848 ↗First posted 2 May 2025cs.CY cs.AI cs.CLcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Medical LLM alignment

Abstract (arXiv)

The wide exploration of large language models (LLMs) raises the awareness of alignment between healthcare stakeholder preferences and model outputs. This alignment becomes a crucial foundation to empower the healthcare workflow effectively, safely, and responsibly. Yet the varying behaviors of LLMs may not always match with healthcare stakeholders' knowledge, demands, and values. To enable a human-AI alignment, healthcare stakeholders will need to perform essential roles in guiding and enhancing the performance of LLMs. Human professionals must participate in the entire life cycle of adopting LLM in healthcare, including training data curation, model training, and inference. In this review, we discuss the approaches, tools, and applications of alignments between healthcare stakeholders and LLMs. We demonstrate that LLMs can better follow human values by properly enhancing healthcare knowledge integration, task understanding, and human guidance. We provide outlooks on enhancing the alignment between humans and LLMs to build trustworthy real-world healthcare applications.

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in Medical LLM alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 43 of its 75 references that are in the map. First posted on arXiv 2 May 2025.

Builds on

43 of 75 references.

References · 43

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2023GPT-4 Technical Reportcited by 18,278
  3. 2022Training language models to follow instructions with human feedbackcited by 15,856
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,072
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,657
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,317
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,068
  9. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,942
  10. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,409
  11. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,001
  12. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 760
31 more
  1. 2022Galactica: A Large Language Model for Sciencecited by 741
  2. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 717
  3. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 617
  4. 2019ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmissioncited by 490
  5. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 459
  6. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 401
  7. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 285
  8. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 272
  9. 2023Large Language Model Alignment: A Surveycited by 243
  10. 2022GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Recordscited by 238
  11. 2022Can large language models reason about medical questions?cited by 194
  12. 2023A Study of Generative Large Language Model for Medical Research and Healthcarecited by 162
  13. 2021MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcarecited by 159
  14. 2024Continual Learning for Large Language Models: A Surveycited by 144
  15. 2023On the Challenges and Perspectives of Foundation Models for Medical Image Analysiscited by 135
  16. 2020ProGen: Language Modeling for Protein Generationcited by 128
  17. 2023Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicinecited by 83
  18. 2023Matching Patients to Clinical Trials with Large Language Modelscited by 69
  19. 2023Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinationscited by 63
  20. 2023Radiology-Llama2: Best-in-Class Large Language Model for Radiologycited by 42
  21. 2023Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domaincited by 38
  22. 2024Data-Centric Foundation Models in Computational Healthcare: A Surveycited by 27
  23. 2023On the Safety of Open-Sourced Large Language Models: Does Alignment Really Prevent Them From Being Misused?cited by 26
  24. 2024Aligning Human Knowledge with Visual Concepts Towards Explainable Medical Image Classificationcited by 18
  25. 2025RadAlign: Advancing Radiology Report Generation with Vision-Language Concept Alignmentcited by 12
  26. 2023AutoTrial: Prompting Language Models for Clinical Trial Designcited by 11
  27. 2023CliniDigest: A Case Study in Large Language Model Based Large-Scale Summarization of Clinical Trial Descriptionscited by 10
  28. 2023Improving Patient Pre-screening for Clinical Trials: Assisting Physicians with Large Language Modelscited by 6
  29. 2023covLLM: Large Language Models for COVID-19 Biomedical Literaturecited by 2
  30. 2021Neural Language Models with Distant Supervision to Identify Major Depressive Disorder from Clinical Notescited by 1
  31. 2025MedForge: Building Medical Foundation Models Like Open Source Software Developmentcited by 1

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Can "AI" Be a Doctor? A Study of Empathy, Readability, and Alignment in Clinical LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review
  2. 2025VITAL: A New Dataset for Benchmarking Pluralistic Alignment in Healthcare
  3. 2026Alignment Plausibility: A New Standard for Assuring AI in Healthcare
  4. 2025Enabling Doctor-Centric Medical AI with LLMs through Workflow-Aligned Tasks and Benchmarks
  5. 2024MedCare: Advancing Medical LLMs through Decoupling Clinical Alignment and Knowledge Aggregation
  6. 2025Evaluating AI Alignment in LLMs: Output Analysis of Value Priorities Across 75 Models with Human Benchmarking
  7. 2026A Human-Centric Pipeline for Aligning Large Language Models with Chinese Medical Ethics
  8. 2026LLM-as-a-Judge in Healthcare: A Scoping Analysis of Applications, Methods, and Human Alignment