mlsanityData to 7 Oct 2026

Aligning Large Language Models for Clinical Tasks

arXiv 2309.02884 ↗First posted 6 Sep 2023cs.CLcited by 6

On the map: LLM alignment / RLHF / preference optimisation → Medical LLM alignment

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable adaptability, showcasing their capacity to excel in tasks for which they were not explicitly trained. However, despite their impressive natural language processing (NLP) capabilities, effective alignment of LLMs remains a crucial challenge when deploying them for specific clinical applications. The ability to generate responses with factually accurate content and to engage in non-trivial reasoning steps are crucial for the LLMs to be eligible for applications in clinical medicine. Employing a combination of techniques including instruction-tuning and in-prompt strategies like few-shot and chain-of-thought prompting has significantly enhanced the performance of LLMs. Our proposed alignment strategy for medical question-answering, known as 'expand-guess-refine', offers a parameter and data-efficient solution. A preliminary analysis of this method demonstrated outstanding performance, achieving a score of 70.63% on a subset of questions sourced from the USMLE dataset.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Medical LLM alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 35 of its 55 references that are in the map. First posted on arXiv 6 Sep 2023.

Builds on

35 of 55 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  11. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  12. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
23 more
  1. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  2. 2017Billion-scale similarity search with GPUscited by 2,477
  3. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  4. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  5. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  6. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  7. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  8. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  9. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  10. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  11. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  12. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  13. 2022Galactica: A Large Language Model for Sciencecited by 736
  14. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  15. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  16. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  17. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 382
  18. 2022Can large language models reason about medical questions?cited by 194
  19. 2021The Factual Inconsistency Problem in Abstractive Text Summarization: A Surveycited by 79
  20. 2020BioMegatron: Larger Biomedical Domain Language Modelcited by 76
  21. 2022Large Language Models and the Reverse Turing Testcited by 38
  22. 2022The Diminishing Returns of Masked Language Models to Sciencecited by 22
  23. 2023Retrieval Augmented Generation and Representative Vector Summarization for large unstructured textual data in Medical Educationcited by 10

Built on by

Cited by year

Citing papers per year: 2024: 5; 2025: 1.

6 papers.

  1. 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
  2. 2024Aligning (Medical) LLMs for (Counterfactual) Fairnesscited by 7
  3. 2024Evaluating Fine-Tuning Efficiency of Human-Inspired Learning Strategies in Medical Question Answeringcited by 2
  4. 2024On the Reliability of Large Language Models to Misinformed and Demographically-Informed Promptscited by 2
  5. 2024LanguaShrink: Reducing Token Overhead with Psycholinguisticscited by 0
  6. 2025MedOrchestra: A Hybrid Cloud-Local LLM Approach for Clinical Data Interpretationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026To Adapt or not to Adapt, Rethinking the Value of Medical Knowledge-Aware Large Language Models
  2. 2024The Limited Impact of Medical Adaptation of Large Language and Vision-Language Models
  3. 2024MedCare: Advancing Medical LLMs through Decoupling Clinical Alignment and Knowledge Aggregation
  4. 2024A Survey for Large Language Models in Biomedicine
  5. 2025Collaboration among Multiple Large Language Models for Medical Question Answering
  6. 2025Structured Outputs Enable General-Purpose LLMs to be Medical Experts
  7. 2024Medical Adaptation of Large Language and Vision-Language Models: Are We Making Progress?
  8. 2024LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Models