mlsanityData to 14 Oct 2026

Abstractive summarization of hospitalisation histories with transformer networks

arXiv 2204.02208 ↗First posted 5 Apr 2022cs.CL cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper we present a novel approach to abstractive summarization of patient hospitalisation histories. We applied an encoder-decoder framework with Longformer neural network as an encoder and BERT as a decoder. Our experiments show improved quality on some summarization tasks compared with pointer-generator networks. We also conducted a study with experienced physicians evaluating the results of our model in comparison with PGN baseline and human-generated abstracts, which showed the effectiveness of our model.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 17 of its 20 references that are in the map. First posted on arXiv 5 Apr 2022.

Builds on

17 of 20 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2020Longformer: The Long-Document Transformercited by 3,413
  6. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  7. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  8. 2020Reformer: The Efficient Transformercited by 1,668
  9. 2020Rethinking Attention with Performerscited by 1,601
  10. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
  11. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  12. 2016Training Deep Nets with Sublinear Memory Costcited by 848
5 more
  1. 2020ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-trainingcited by 222
  2. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
  3. 2018Learning to Summarize Radiology Findingscited by 54
  4. 2020Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structurescited by 49
  5. 2019Topic-aware Pointer-Generator Networks for Summarizing Spoken Conversationscited by 41

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 1.

4 papers.

  1. 2023A Method to Automate the Discharge Summary Hospital Course for Neurology Patientscited by 6
  2. 2023A Survey for Biomedical Text Summarization: From Pre-trained to Large Language Modelscited by 3
  3. 2024Summarizing Radiology Reports Findings into Impressionscited by 1
  4. 2023Diagnosing Transformers: Illuminating Feature Spaces for Clinical Decision-Makingcited by 0