# Enhancing Summarization Performance through Transformer-Based Prompt Engineering in Automated Medical Reporting

arXiv: https://arxiv.org/abs/2311.13274 · HTML: https://mlsanity.com/paper/2311.13274/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 60 references that are in the map.
- First posted on arXiv 22 Nov 2023.

## Builds on (most cited first, 19 of 19)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning](https://mlsanity.com/paper/2205.03401/) (2022)
- [Generating SOAP Notes from Doctor-Patient Conversations Using Modular Summarization Techniques](https://mlsanity.com/paper/2005.01795/) (2020)
- [Prompt Engineering for Healthcare: Methodologies and Applications](https://mlsanity.com/paper/2304.14670/) (2023)
- [Human Evaluation and Correlation with Automatic Metrics in Consultation Note Generation](https://mlsanity.com/paper/2204.00447/) (2022)
- [Benchmarking LLM powered Chatbots: Methods and Metrics](https://mlsanity.com/paper/2308.04624/) (2023)
- [Automatic Text Summarization Methods: A Comprehensive Review](https://mlsanity.com/paper/2204.01849/) (2022)
- [A Survey on Medical Document Summarization](https://mlsanity.com/paper/2212.01669/) (2022)
- [SummQA at MEDIQA-Chat 2023:In-Context Learning with GPT-4 for Medical Summarization](https://mlsanity.com/paper/2306.17384/) (2023)
- [Consultation Checklists: Standardising the Human Evaluation of Medical Note Generation](https://mlsanity.com/paper/2211.09455/) (2022)
- [Generating medically-accurate summaries of patient-provider dialogue: A multi-stage approach using large language models](https://mlsanity.com/paper/2305.05982/) (2023)
- [A Semi-automated Method for Domain-Specific Ontology Creation from Medical Guidelines](https://mlsanity.com/paper/2111.15338/) (2021)
- [Prompt Engineering For Students of Medicine and Their Teachers](https://mlsanity.com/paper/2308.11628/) (2023)

## Built on by (most cited first, 3 of 3)

- [Efficient Fine-Tuning of Large Language Models for Automated Medical Documentation](https://mlsanity.com/paper/2409.09324/) (2024)
- [Prompt engineering paradigms for medical applications: scoping review and recommendations for better practices](https://mlsanity.com/paper/2405.01249/) (2024)
- [Generating High Quality Synthetic Data for Dutch Medical Conversations](https://mlsanity.com/paper/2604.09645/) (2026)

## Abstract (arXiv)

Customized medical prompts enable Large Language Models (LLM) to effectively address medical dialogue summarization. The process of medical reporting is often time-consuming for healthcare professionals. Implementing medical dialogue summarization techniques presents a viable solution to alleviate this time constraint by generating automated medical reports. The effectiveness of LLMs in this process is significantly influenced by the formulation of the prompt, which plays a crucial role in determining the quality and relevance of the generated reports. In this research, we used a combination of two distinct prompting strategies, known as shot prompting and pattern prompting to enhance the performance of automated medical reporting. The evaluation of the automated medical reports is carried out using the ROUGE score and a human evaluation with the help of an expert panel. The two-shot prompting approach in combination with scope and domain context outperforms other methods and achieves the highest score when compared to the human reference set by a general practitioner. However, the automated reports are approximately twice as long as the human references, due to the addition of both redundant and relevant statements that are added to the report.