# Automatic Text Summarization of COVID-19 Medical Research Articles using BERT and GPT-2

arXiv: https://arxiv.org/abs/2006.01997 · HTML: https://mlsanity.com/paper/2006.01997/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2020.

## Built on by (most cited first, 23 of 23)

- [Pre-trained Language Models in Biomedical Domain: A Systematic Survey](https://mlsanity.com/paper/2110.05006/) (2021)
- [Endora: Video Generation Models as Endoscopy Simulators](https://mlsanity.com/paper/2403.11050/) (2024)
- [SentimentGPT: Exploiting GPT for Advanced Sentiment Analysis and its Departure from Current Machine Learning](https://mlsanity.com/paper/2307.10234/) (2023)
- [Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbots](https://mlsanity.com/paper/2209.03463/) (2022)
- [IC3: Image Captioning by Committee Consensus](https://mlsanity.com/paper/2302.01328/) (2023)
- [Abstractive Text Summarization: State of the Art, Challenges, and Improvements](https://mlsanity.com/paper/2409.02413/) (2024)
- [Guiding Generative Language Models for Data Augmentation in Few-Shot Text Classification](https://mlsanity.com/paper/2111.09064/) (2021)
- [A Survey on Medical Document Summarization](https://mlsanity.com/paper/2212.01669/) (2022)
- [A Comparative Study of Quality Evaluation Methods for Text Summarization](https://mlsanity.com/paper/2407.00747/) (2024)
- [Fine-tuning GPT-3 for Russian Text Summarization](https://mlsanity.com/paper/2108.03502/) (2021)
- [CliniDigest: A Case Study in Large Language Model Based Large-Scale Summarization of Clinical Trial Descriptions](https://mlsanity.com/paper/2307.14522/) (2023)
- [Gradient-Free Privacy Leakage in Federated Language Models through Selective Weight Tampering](https://mlsanity.com/paper/2310.16152/) (2023)
- [Automatic Controllable Product Copywriting for E-Commerce](https://mlsanity.com/paper/2206.10103/) (2022)
- [From Visual Prompt Learning to Zero-Shot Transfer: Mapping Is All You Need](https://mlsanity.com/paper/2303.05266/) (2023)
- [A Survey for Biomedical Text Summarization: From Pre-trained to Large Language Models](https://mlsanity.com/paper/2304.08763/) (2023)
- [Automatic Generation of Factual News Headlines in Finnish](https://mlsanity.com/paper/2212.02170/) (2022)
- [Abstractive Text Summarization Using the BRIO Training Paradigm](https://mlsanity.com/paper/2305.13696/) (2023)
- [Deep Transfer Learning Based Peer Review Aggregation and Meta-review Generation for Scientific Articles](https://mlsanity.com/paper/2410.04202/) (2024)
- [CSTRL: Context-Driven Sequential Transfer Learning for Abstractive Radiology Report Summarization](https://mlsanity.com/paper/2503.05750/) (2025)
- [GeSERA: General-domain Summary Evaluation by Relevance Analysis](https://mlsanity.com/paper/2110.03567/) (2021)
- [Nested Named-Entity Recognition on Vietnamese COVID-19: Dataset and Experiments](https://mlsanity.com/paper/2504.21016/) (2025)
- [Clinically Interpretable Sepsis Early Warning via LLM-Guided Simulation of Temporal Physiological Dynamics](https://mlsanity.com/paper/2604.20924/) (2026)
- [BadBone: Backdoor Attacks Against Backbone Models in Visual Prompt Learning](https://mlsanity.com/paper/2605.31246/) (2026)

## Abstract (arXiv)

With the COVID-19 pandemic, there is a growing urgency for medical community to keep up with the accelerating growth in the new coronavirus-related literature. As a result, the COVID-19 Open Research Dataset Challenge has released a corpus of scholarly articles and is calling for machine learning approaches to help bridging the gap between the researchers and the rapidly growing publications. Here, we take advantage of the recent advances in pre-trained NLP models, BERT and OpenAI GPT-2, to solve this challenge by performing text summarization on this dataset. We evaluate the results using ROUGE scores and visual inspection. Our model provides abstractive and comprehensive information based on keywords extracted from the original articles. Our work can help the the medical community, by providing succinct summaries of articles for which the abstract are not already available.