# SPeC: A Soft Prompt-Based Calibration on Performance Variability of Large Language Model in Clinical Notes Summarization

arXiv: https://arxiv.org/abs/2303.13035 · HTML: https://mlsanity.com/paper/2303.13035/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 23 Mar 2023.

## Builds on (most cited first, 25 of 25)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://mlsanity.com/paper/2010.15980/) (2020)
- [SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer](https://mlsanity.com/paper/2110.07904/) (2021)
- [Reframing Instructional Prompts to GPTk's Language](https://mlsanity.com/paper/2109.07830/) (2021)
- [Ask Me Anything: A simple strategy for prompting language models](https://mlsanity.com/paper/2210.02441/) (2022)
- [Neural Prompt Search](https://mlsanity.com/paper/2206.04673/) (2022)
- [PromptChainer: Chaining Large Language Model Prompts through Visual Programming](https://mlsanity.com/paper/2203.06566/) (2022)
- [MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcare](https://mlsanity.com/paper/1810.09593/) (2018)
- [PromptEHR: Conditional Electronic Healthcare Records Generation with Prompt Learning](https://mlsanity.com/paper/2211.01761/) (2022)
- [A Survey on Medical Document Summarization](https://mlsanity.com/paper/2212.01669/) (2022)
- [Prompt Tuning of Deep Neural Networks for Speaker-adaptive Visual Speech Recognition](https://mlsanity.com/paper/2302.08102/) (2023)
- [PheME: A deep ensemble framework for improving phenotype prediction from multi-modal data](https://mlsanity.com/paper/2303.10794/) (2023)

## Built on by (most cited first, 11 of 11)

- [A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods](https://mlsanity.com/paper/2403.02901/) (2024)
- [Prompt Engineering for Healthcare: Methodologies and Applications](https://mlsanity.com/paper/2304.14670/) (2023)
- [LongHealth: A Question Answering Benchmark with Long Clinical Documents](https://mlsanity.com/paper/2401.14490/) (2024)
- [Assessing and Enhancing Large Language Models in Rare Disease Question-answering](https://mlsanity.com/paper/2408.08422/) (2024)
- [DHP Benchmark: Are LLMs Good NLG Evaluators?](https://mlsanity.com/paper/2408.13704/) (2024)
- [Unifying Corroborative and Contributive Attributions in Large Language Models](https://mlsanity.com/paper/2311.12233/) (2023)
- [SPEER: Sentence-Level Planning of Long Clinical Summaries via Embedded Entity Retrieval](https://mlsanity.com/paper/2401.02369/) (2024)
- [Utilizing ChatGPT to Enhance Clinical Trial Enrollment](https://mlsanity.com/paper/2306.02077/) (2023)
- [Enhanced Electronic Health Records Text Summarization Using Large Language Models](https://mlsanity.com/paper/2410.09628/) (2024)
- [Assessing the role of clinical summarization and patient chart review within communications, medical management, and diagnostics](https://mlsanity.com/paper/2407.16905/) (2024)
- [Comparative Analysis of Abstractive Summarization Models for Clinical Radiology Reports](https://mlsanity.com/paper/2506.16247/) (2025)

## Abstract (arXiv)

Electronic health records (EHRs) store an extensive array of patient information, encompassing medical histories, diagnoses, treatments, and test outcomes. These records are crucial for enabling healthcare providers to make well-informed decisions regarding patient care. Summarizing clinical notes further assists healthcare professionals in pinpointing potential health risks and making better-informed decisions. This process contributes to reducing errors and enhancing patient outcomes by ensuring providers have access to the most pertinent and current patient data. Recent research has shown that incorporating prompts with large language models (LLMs) substantially boosts the efficacy of summarization tasks. However, we show that this approach also leads to increased output variance, resulting in notably divergent outputs even when prompts share similar meanings. To tackle this challenge, we introduce a model-agnostic Soft Prompt-Based Calibration (SPeC) pipeline that employs soft prompts to diminish variance while preserving the advantages of prompt-based summarization. Experimental findings on multiple clinical note tasks and LLMs indicate that our method not only bolsters performance but also effectively curbs variance for various LLMs, providing a more uniform and dependable solution for summarizing vital medical information.