# Continual BERT: Continual Learning for Adaptive Extractive Summarization of COVID-19 Literature

arXiv: https://arxiv.org/abs/2007.03405 · HTML: https://mlsanity.com/paper/2007.03405/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jul 2020.

## Built on by (most cited first, 6 of 6)

- [On the Evaluation of Machine-Generated Reports](https://mlsanity.com/paper/2405.00982/) (2024)
- [A Survey on Medical Document Summarization](https://mlsanity.com/paper/2212.01669/) (2022)
- [Exploring the Efficacy of Large Language Models in Summarizing Mental Health Counseling Sessions: A Benchmark Study](https://mlsanity.com/paper/2402.19052/) (2024)
- [RewardsOfSum: Exploring Reinforcement Learning Rewards for Summarisation](https://mlsanity.com/paper/2106.04080/) (2021)
- [A Survey for Biomedical Text Summarization: From Pre-trained to Large Language Models](https://mlsanity.com/paper/2304.08763/) (2023)
- [BERT WEAVER: Using WEight AVERaging to enable lifelong learning for transformer-based models in biomedical semantic search engines](https://mlsanity.com/paper/2202.10101/) (2022)

## Abstract (arXiv)

The scientific community continues to publish an overwhelming amount of new research related to COVID-19 on a daily basis, leading to much literature without little to no attention. To aid the community in understanding the rapidly flowing array of COVID-19 literature, we propose a novel BERT architecture that provides a brief yet original summarization of lengthy papers. The model continually learns on new data in online fashion while minimizing catastrophic forgetting, thus fitting to the need of the community. Benchmark and manual examination of its performance show that the model provide a sound summary of new scientific literature.