# An Analysis of Abstractive Text Summarization Using Pre-trained Models

arXiv: https://arxiv.org/abs/2303.12796 · HTML: https://mlsanity.com/paper/2303.12796/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 22 references that are in the map.
- First posted on arXiv 25 Feb 2023.

## Builds on (most cited first, 14 of 14)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization](https://mlsanity.com/paper/1911.12237/) (2019)
- [BillSum: A Corpus for Automatic Summarization of US Legislation](https://mlsanity.com/paper/1910.00523/) (2019)
- [Neural Abstractive Text Summarization with Sequence-to-Sequence Models](https://mlsanity.com/paper/1812.02303/) (2018)
- [Global Encoding for Abstractive Summarization](https://mlsanity.com/paper/1805.03989/) (2018)
- [Abstractive Text Summarization based on Language Model Conditioning and Locality Modeling](https://mlsanity.com/paper/2003.13027/) (2020)

## Built on by (most cited first, 7 of 7)

- [Hallucination Reduction in Long Input Text Summarization](https://mlsanity.com/paper/2309.16781/) (2023)
- [Evaluating LLMs and Pre-trained Models for Text Summarization Across Diverse Datasets](https://mlsanity.com/paper/2502.19339/) (2025)
- [Analysis of Multidomain Abstractive Summarization Using Salience Allocation](https://mlsanity.com/paper/2402.11955/) (2024)
- [How Green are Neural Language Models? Analyzing Energy Consumption in Text Summarization Fine-tuning](https://mlsanity.com/paper/2501.15398/) (2025)
- [Abstractive Text Summarization for Resumes With Cutting Edge NLP Transformers and LSTM](https://mlsanity.com/paper/2306.13315/) (2023)
- [Controlled Randomness Improves the Performance of Transformer Models](https://mlsanity.com/paper/2310.13526/) (2023)
- [Comparative Analysis of Abstractive Summarization Models for Clinical Radiology Reports](https://mlsanity.com/paper/2506.16247/) (2025)

## Abstract (arXiv)

People nowadays use search engines like Google, Yahoo, and Bing to find information on the Internet. Due to explosion in data, it is helpful for users if they are provided relevant summaries of the search results rather than just links to webpages. Text summarization has become a vital approach to help consumers swiftly grasp vast amounts of information.In this paper, different pre-trained models for text summarization are evaluated on different datasets. Specifically, we have used three different pre-trained models, namely, google/pegasus-cnn-dailymail, T5-base, facebook/bart-large-cnn. We have considered three different datasets, namely, CNN-dailymail, SAMSum and BillSum to get the output from the above three models. The pre-trained models are compared over these different datasets, each of 2000 examples, through ROUGH and BLEU metrics.