# Summarizing Patients Problems from Hospital Progress Notes Using Pre-trained Sequence-to-Sequence Models

arXiv: https://arxiv.org/abs/2208.08408 · HTML: https://mlsanity.com/paper/2208.08408/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 49 references that are in the map.
- First posted on arXiv 17 Aug 2022.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Self-Alignment Pretraining for Biomedical Entity Representations](https://mlsanity.com/paper/2010.11784/) (2020)
- [Aequitas: A Bias and Fairness Audit Toolkit](https://mlsanity.com/paper/1811.05577/) (2018)
- [Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?](https://mlsanity.com/paper/2005.00628/) (2020)
- [Generating SOAP Notes from Doctor-Patient Conversations Using Modular Summarization Techniques](https://mlsanity.com/paper/2005.01795/) (2020)
- [Learning to Summarize Radiology Findings](https://mlsanity.com/paper/1809.04698/) (2018)
- [Leveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversations](https://mlsanity.com/paper/2109.12174/) (2021)
- [What's in a Summary? Laying the Groundwork for Advances in Hospital-Course Summarization](https://mlsanity.com/paper/2105.00816/) (2021)
- [Ontology-Aware Clinical Abstractive Summarization](https://mlsanity.com/paper/1905.05818/) (2019)
- [Attend to Medical Ontologies: Content Selection for Clinical Abstractive Summarization](https://mlsanity.com/paper/2005.00163/) (2020)
- [Boosting Low-Resource Biomedical QA via Entity-Aware Masking Strategies](https://mlsanity.com/paper/2102.08366/) (2021)
- [Towards objectively evaluating the quality of generated medical summaries](https://mlsanity.com/paper/2104.04412/) (2021)

## Built on by (most cited first, 30 of 30)

- [Leveraging Medical Knowledge Graphs Into Large Language Models for Diagnosis Prediction: Design and Application Study](https://mlsanity.com/paper/2308.14321/) (2023)
- [MEDIC: Comprehensive Evaluation of Leading Indicators for LLM Safety and Utility in Clinical Applications](https://mlsanity.com/paper/2409.07314/) (2024)
- [Biomedical Large Languages Models Seem not to be Superior to Generalist Models on Unseen Medical Data](https://mlsanity.com/paper/2408.13833/) (2024)
- [DR.BENCH: Diagnostic Reasoning Benchmark for Clinical Natural Language Processing](https://mlsanity.com/paper/2209.14901/) (2022)
- [Exploring the Efficacy of Large Language Models in Summarizing Mental Health Counseling Sessions: A Benchmark Study](https://mlsanity.com/paper/2402.19052/) (2024)
- [Does Biomedical Training Lead to Better Medical Performance?](https://mlsanity.com/paper/2404.04067/) (2024)
- [Overview of the Problem List Summarization (ProbSum) 2023 Shared Task on Summarizing Patients' Active Diagnoses and Problems from Electronic Health Record Progress Notes](https://mlsanity.com/paper/2306.05270/) (2023)
- [CUED at ProbSum 2023: Hierarchical Ensemble of Summarization Models](https://mlsanity.com/paper/2306.05317/) (2023)
- [Generating medically-accurate summaries of patient-provider dialogue: A multi-stage approach using large language models](https://mlsanity.com/paper/2305.05982/) (2023)
- [A Method to Automate the Discharge Summary Hospital Course for Neurology Patients](https://mlsanity.com/paper/2305.06416/) (2023)
- [PULSAR: Pre-training with Extracted Healthcare Terms for Summarising Patients' Problems and Data Augmentation with Black-box Large Language Models](https://mlsanity.com/paper/2306.02754/) (2023)
- [Shimo Lab at "Discharge Me!": Discharge Summarization by Prompt-Driven Concatenation of Electronic Health Record Sections](https://mlsanity.com/paper/2406.18094/) (2024)
- [Evaluation of Large Language Models for Summarization Tasks in the Medical Domain: A Narrative Review](https://mlsanity.com/paper/2409.18170/) (2024)
- [Natural Language Generation in Healthcare: A Review of Methods and Applications](https://mlsanity.com/paper/2505.04073/) (2025)
- [Zero-shot and Few-shot Generation Strategies for Artificial Clinical Records](https://mlsanity.com/paper/2403.08664/) (2024)
- [IgnitionInnovators at "Discharge Me!": Chain-of-Thought Instruction Finetuning Large Language Models for Discharge Summaries](https://mlsanity.com/paper/2407.17636/) (2024)
- [A Scoping Review of Natural Language Processing in Addressing Medically Inaccurate Information: Errors, Misinformation, and Hallucination](https://mlsanity.com/paper/2505.00008/) (2025)
- [Beyond Literal Summarization: Redefining Hallucination for Medical SOAP Note Evaluation](https://mlsanity.com/paper/2604.14829/) (2026)
- [Personalized Federated Learning via Gradient Modulation for Heterogeneous Text Summarization](https://mlsanity.com/paper/2304.11524/) (2023)
- [NOTE: Notable generation Of patient Text summaries through Efficient approach based on direct preference optimization](https://mlsanity.com/paper/2402.11882/) (2024)
- [Privacy-Preserving Generation of Clinical Narratives from Medical Terminologies](https://mlsanity.com/paper/2509.10882/) (2025)
- [Query-Guided Self-Supervised Summarization of Nursing Notes](https://mlsanity.com/paper/2407.04125/) (2024)
- [Abstract Meaning Representation for Hospital Discharge Summarization](https://mlsanity.com/paper/2506.14101/) (2025)
- [Countering Catastrophic Forgetting of Large Language Models for Better Instruction Following via Weight-Space Model Merging](https://mlsanity.com/paper/2604.01538/) (2026)
- [Systematic Evaluation of the Quality of Synthetic Clinical Notes Rephrased by LLMs at Million-Note Scale](https://mlsanity.com/paper/2605.17775/) (2026)
- [ClinicalMC: A Benchmark for Multi-Course Clinical Decision-Making with Large Language Models](https://mlsanity.com/paper/2606.03157/) (2026)
- [MedGuards: Multi-Agent System for Reliable Medical Error Detection and Correction](https://mlsanity.com/paper/2606.25651/) (2026)
- [Surgical Alignment in Knowledge Graph Training for Clinical Diagnosis with Large Language Models](https://mlsanity.com/paper/2608.26587/) (2026)
- [Toward Complete Hospital Discharge Summarization with Abstract Meaning Representation](https://mlsanity.com/paper/2609.13581/) (2026)
- [CliniCIRCA: A Modular LLM Framework for Constructing Longitudinal Mental Health Patient Journeys from Raw EHR Narratives](https://mlsanity.com/paper/2609.19585/) (2026)

## Abstract (arXiv)

Automatically summarizing patients' main problems from daily progress notes using natural language processing methods helps to battle against information and cognitive overload in hospital settings and potentially assists providers with computerized diagnostic decision support. Problem list summarization requires a model to understand, abstract, and generate clinical documentation. In this work, we propose a new NLP task that aims to generate a list of problems in a patient's daily care plan using input from the provider's progress notes during hospitalization. We investigate the performance of T5 and BART, two state-of-the-art seq2seq transformer architectures, in solving this problem. We provide a corpus built on top of progress notes from publicly available electronic health record progress notes in the Medical Information Mart for Intensive Care (MIMIC)-III. T5 and BART are trained on general domain text, and we experiment with a data augmentation method and a domain adaptation pre-training method to increase exposure to medical vocabulary and knowledge. Evaluation methods include ROUGE, BERTScore, cosine similarity on sentence embedding, and F-score on medical concepts. Results show that T5 with domain adaptive pre-training achieves significant performance gains compared to a rule-based system and general domain pre-trained language models, indicating a promising direction for tackling the problem summarization task.