# Bayesian Active Summarization

arXiv: https://arxiv.org/abs/2110.04480 · HTML: https://mlsanity.com/paper/2110.04480/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 36 references that are in the map.
- First posted on arXiv 9 Oct 2021.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies](https://mlsanity.com/paper/1804.11283/) (2018)
- [Deep Active Learning for Named Entity Recognition](https://mlsanity.com/paper/1707.05928/) (2017)
- [Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study](https://mlsanity.com/paper/1808.05697/) (2018)
- [Wat zei je? Detecting Out-of-Distribution Translations with Variational Transformers](https://mlsanity.com/paper/2006.08344/) (2020)
- [Active Learning with Partial Feedback](https://mlsanity.com/paper/1802.07427/) (2018)
- [Fine-tuning BERT for Low-Resource Natural Language Understanding via Active Learning](https://mlsanity.com/paper/2012.02462/) (2020)
- [On the Importance of Strong Baselines in Bayesian Deep Learning](https://mlsanity.com/paper/1811.09385/) (2018)
- [Should We Trust This Summary? Bayesian Abstractive Summarization to The Rescue](https://mlsanity.com/paper/2105.10155/) (2021)
- [LTP: A New Active Learning Strategy for CRF-Based Named Entity Recognition](https://mlsanity.com/paper/2001.02524/) (2020)

## Built on by (most cited first, 5 of 5)

- [Active Learning for Abstractive Text Summarization](https://mlsanity.com/paper/2301.03252/) (2023)
- [Active Learning for Natural Language Generation](https://mlsanity.com/paper/2305.15040/) (2023)
- [Can We Trust the Performance Evaluation of Uncertainty Estimation Methods in Text Summarization?](https://mlsanity.com/paper/2406.17274/) (2024)
- [DUAL: Diversity and Uncertainty Active Learning for Text Summarization](https://mlsanity.com/paper/2503.00867/) (2025)
- [Combining X-Vectors and Bayesian Batch Active Learning: Two-Stage Active Learning Pipeline for Speech Recognition](https://mlsanity.com/paper/2406.02566/) (2024)

## Abstract (arXiv)

Bayesian Active Learning has had significant impact to various NLP problems, but nevertheless it's application to text summarization has been explored very little. We introduce Bayesian Active Summarization (BAS), as a method of combining active learning methods with state-of-the-art summarization models. Our findings suggest that BAS achieves better and more robust performance, compared to random selection, particularly for small and very small data annotation budgets. Using BAS we showcase it is possible to leverage large summarization models to effectively solve real-world problems with very limited annotated data.