mlsanityData to 7 Oct 2026

Do Physicians Know How to Prompt? The Need for Automatic Prompt Optimization Help in Clinical Note Generation

arXiv 2311.09684 ↗First posted 16 Nov 2023cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

This study examines the effect of prompt engineering on the performance of Large Language Models (LLMs) in clinical note generation. We introduce an Automatic Prompt Optimization (APO) framework to refine initial prompts and compare the outputs of medical experts, non-medical experts, and APO-enhanced GPT3.5 and GPT4. Results highlight GPT4 APO's superior performance in standardizing prompt quality across clinical note sections. A human-in-the-loop approach shows that experts maintain content quality post-APO, with a preference for their own modifications, suggesting the value of expert customization. We recommend a two-phase optimization process, leveraging APO-GPT4 for consistency and expert input for personalization.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 27 of its 54 references that are in the map. First posted on arXiv 16 Nov 2023.

Builds on

27 of 54 references.

References · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2019Language Models as Knowledge Bases?cited by 1,922
  10. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  11. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  12. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
15 more
  1. 2022Large Language Models Can Self-Improvecited by 656
  2. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  3. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  4. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
  5. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  6. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 226
  7. 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 189
  8. 2021Can Language Models be Biomedical Knowledge Bases?cited by 83
  9. 2020Generating SOAP Notes from Doctor-Patient Conversations Using Modular Summarization Techniquescited by 64
  10. 2023NoteChat: A Dataset of Synthetic Doctor-Patient Conversations Conditioned on Clinical Notescited by 48
  11. 2023Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Samplingcited by 46
  12. 2023AutoHint: Automatic Prompt Optimization with Hint Generationcited by 29
  13. 2022Extracting Biomedical Factual Knowledge Using Pretrained Language Model and Electronic Health Record Contextcited by 15
  14. 2022Context Variance Evaluation of Pretrained Language Models for Prompt-based Biomedical Knowledge Probingcited by 15
  15. 2023A Meta-Evaluation of Faithfulness Metrics for Long-Form Hospital-Course Summarizationcited by 14

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2025SynthEHR-Eviction: Enhancing Eviction SDoH Detection with LLM-Augmented Synthetic EHR Datacited by 1
  2. 2024UMass-BioNLP at MEDIQA-M3G 2024: DermPrompt -- A Systematic Exploration of Prompt Engineering with GPT-4V for Dermatological Diagnosiscited by 0