mlsanityData to 14 Oct 2026

Attention-based Clinical Note Summarization

arXiv 2104.08942 ↗First posted 18 Apr 2021cs.CL cs.AI cs.IRcited by 16

Not on the map. Why?

Abstract (arXiv)

In recent years, the trend of deploying digital systems in numerous industries has hiked. The health sector has observed an extensive adoption of digital systems and services that generate significant medical records. Electronic health records contain valuable information for prospective and retrospective analysis that is often not entirely exploited because of the complicated dense information storage. The crude purpose of condensing health records is to select the information that holds most characteristics of the original documents based on a reported disease. These summaries may boost diagnosis and save a doctor's time during a saturated workload situation like the COVID-19 pandemic. In this paper, we are applying a multi-head attention-based mechanism to perform extractive summarization of meaningful phrases on clinical notes. Our method finds major sentences for a summary by correlating tokens, segments, and positional embeddings of sentences in a clinical note. The model outputs attention scores that are statistically transformed to extract critical phrases for visualization on the heat-mapping tool and for human use.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 20 of its 60 references that are in the map. First posted on arXiv 18 Apr 2021.

Builds on

20 of 60 references.

References · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  5. 2018Deep contextualized word representationscited by 2,290
  6. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  7. 2019Publicly Available Clinical BERT Embeddingscited by 1,002
  8. 2019Revealing the Dark Secrets of BERTcited by 301
  9. 2019A Multiscale Visualization of Attention in the Transformer Modelcited by 292
  10. 2018Explainable Prediction of Medical Codes from Clinical Textcited by 234
  11. 2015Semi-supervised Sequence Learningcited by 198
  12. 2020Extractive Summarization as Text Matchingcited by 189
8 more
  1. 2019Fine-tune BERT for Extractive Summarizationcited by 122
  2. 2019Leveraging BERT for Extractive Text Summarization on Lecturescited by 63
  3. 2019How Does BERT Answer Questions? A Layer-Wise Analysis of Transformer Representationscited by 63
  4. 2017Text Summarization Techniques: A Brief Surveycited by 59
  5. 2017Graph-based Neural Multi-Document Summarizationcited by 54
  6. 2018Universal Language Model Fine-tuning for Text Classificationcited by 52
  7. 2018Extractive Summarization of EHR Discharge Notescited by 12
  8. 2020A Qualitative Evaluation of Language Models on Automatic Question-Answering for COVID-19cited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 0; 2023: 4; 2024: 5; 2025: 4; 2026: 2.

16 papers.

Sort:

By area:

  1. 2021Pre-trained Language Models in Biomedical Domain: A Systematic Surveycited by 87
  2. 2023Leveraging Medical Knowledge Graphs Into Large Language Models for Diagnosis Prediction: Design and Application Studycited by 30
  3. 2023Vision Transformers for Small Histological Datasets Learned through Knowledge Distillationcited by 10
  4. 2024MEDVOC: Vocabulary Adaptation for Fine-tuning Pre-trained Language Models on Medical Text Summarizationcited by 8
  5. 2025Bridging Simplicity and Sophistication using GLinear: A Novel Architecture for Enhanced Time Series Predictioncited by 4
  6. 2023A Survey for Biomedical Text Summarization: From Pre-trained to Large Language Modelscited by 3
  7. 2023Neural Summarization of Electronic Health Recordscited by 3
  8. 2024Onco-Retriever: Generative Classifier for Retrieval of EHR Records in Oncologycited by 3
  9. 2024Ethical Challenges and Evolving Strategies in the Integration of Artificial Intelligence into Clinical Practicecited by 2
  10. 2024Health Text Simplification: An Annotated Corpus for Digestive Cancer Education and Novel Strategies for Reinforcement Learningcited by 1
  11. 2025FHRFormer: A Self-supervised Transformer Approach for Fetal Heart Rate Inpainting and Forecastingcited by 1
  12. 2024Query-Guided Self-Supervised Summarization of Nursing Notescited by 0
4 more
  1. 2025REMEMBER: Retrieval-based Explainable Multimodal Evidence-guided Modeling for Brain Evaluation and Reasoning in Zero- and Few-shot Neurodegenerative Diagnosiscited by 0
  2. 2025DENSE: Longitudinal Progress Note Generation with Temporal Modeling of Heterogeneous Clinical Notes Across Hospital Visitscited by 0
  3. 2026FHRFormer: A Self-Supervised Masked Transformer Framework for Fetal Heart Rate Time-Series Inpainting and Forecastingcited by 0
  4. 2026Evaluating AI Generated Summaries for Cancer Patientscited by 0