# Learning by Semantic Similarity Makes Abstractive Summarization Better

arXiv: https://arxiv.org/abs/2002.07767 · HTML: https://mlsanity.com/paper/2002.07767/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Feb 2020.

## Built on by (most cited first, 7 of 7)

- [Automatic Text Evaluation through the Lens of Wasserstein Barycenters](https://mlsanity.com/paper/2108.12463/) (2021)
- [InfoLM: A New Metric to Evaluate Summarization & Data2Text Generation](https://mlsanity.com/paper/2112.01589/) (2021)
- [A Pseudo-Metric between Probability Distributions based on Depth-Trimmed Regions](https://mlsanity.com/paper/2103.12711/) (2021)
- [How well do you know your summarization datasets?](https://mlsanity.com/paper/2106.11388/) (2021)
- [Controlling keywords and their positions in text generation](https://mlsanity.com/paper/2304.09516/) (2023)
- [Lightweight Safety Guardrails via Synthetic Data and RL-guided Adversarial Training](https://mlsanity.com/paper/2507.08284/) (2025)
- [Evaluation of Abstractive Summarisation Models with Machine Translation in Deliberative Processes](https://mlsanity.com/paper/2110.05847/) (2021)

## Abstract (arXiv)

By harnessing pre-trained language models, summarization models had rapid progress recently. However, the models are mainly assessed by automatic evaluation metrics such as ROUGE. Although ROUGE is known for having a positive correlation with human evaluation scores, it has been criticized for its vulnerability and the gap between actual qualities. In this paper, we compare the generated summaries from recent LM, BART, and the reference summaries from a benchmark dataset, CNN/DM, using a crowd-sourced human evaluation metric. Interestingly, model-generated summaries receive higher scores relative to reference summaries. Stemming from our experimental results, we first argue the intrinsic characteristics of the CNN/DM dataset, the progress of pre-trained language models, and their ability to generalize on the training data. Finally, we share our insights into the model-generated summaries and presents our thought on learning methods for abstractive summarization.