# Evaluating LLMs and Pre-trained Models for Text Summarization Across Diverse Datasets

arXiv: https://arxiv.org/abs/2502.19339 · HTML: https://mlsanity.com/paper/2502.19339/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 26 Feb 2025.

## Builds on (most cited first, 17 of 17)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics](https://mlsanity.com/paper/2207.00939/) (2022)
- [A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models](https://mlsanity.com/paper/2406.11289/) (2024)
- [Generation of Highlights from Research Papers Using Pointer-Generator Networks and SciBERT Embeddings](https://mlsanity.com/paper/2302.07729/) (2023)
- [Named Entity Recognition Based Automatic Generation of Research Highlights](https://mlsanity.com/paper/2303.12795/) (2023)
- [An Analysis of Abstractive Text Summarization Using Pre-trained Models](https://mlsanity.com/paper/2303.12796/) (2023)
- [Hallucination Reduction in Long Input Text Summarization](https://mlsanity.com/paper/2309.16781/) (2023)
- [Abstractive Text Summarization using Attentive GRU based Encoder-Decoder](https://mlsanity.com/paper/2302.13117/) (2023)
- [Analysis of Multidomain Abstractive Summarization Using Salience Allocation](https://mlsanity.com/paper/2402.11955/) (2024)

## Built on by (most cited first, 3 of 3)

- [MAGIC: Multi-Agent Argumentation and Grammar Integrated Critiquer](https://mlsanity.com/paper/2506.13037/) (2025)
- [The Effect of Document Summarization on LLM-Based Relevance Judgments](https://mlsanity.com/paper/2512.05334/) (2025)
- [Which LLM to pick? Online Active Model Selection for Large Language Models](https://mlsanity.com/paper/2610.01592/) (2026)

## Abstract (arXiv)

Text summarization plays a crucial role in natural language processing by condensing large volumes of text into concise and coherent summaries. As digital content continues to grow rapidly and the demand for effective information retrieval increases, text summarization has become a focal point of research in recent years. This study offers a thorough evaluation of four leading pre-trained and open-source large language models: BART, FLAN-T5, LLaMA-3-8B, and Gemma-7B, across five diverse datasets CNN/DM, Gigaword, News Summary, XSum, and BBC News. The evaluation employs widely recognized automatic metrics, including ROUGE-1, ROUGE-2, ROUGE-L, BERTScore, and METEOR, to assess the models' capabilities in generating coherent and informative summaries. The results reveal the comparative strengths and limitations of these models in processing various text types.