# GUMsley: Evaluating Entity Salience in Summarization for 12 English Genres

arXiv: https://arxiv.org/abs/2401.17974 · HTML: https://mlsanity.com/paper/2401.17974/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 31 Jan 2024.

## Builds on (most cited first, 18 of 18)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [Neural Text Summarization: A Critical Evaluation](https://mlsanity.com/paper/1908.08960/) (2019)
- [BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization](https://mlsanity.com/paper/1906.03741/) (2019)
- [Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation](https://mlsanity.com/paper/2212.07981/) (2022)
- [Faithful to the Original: Fact Aware Neural Abstractive Summarization](https://mlsanity.com/paper/1711.04434/) (2017)
- [Entity-level Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/2102.09130/) (2021)
- [Controllable Abstractive Summarization](https://mlsanity.com/paper/1711.05217/) (2017)
- [EntSUM: A Data Set for Entity-Centric Summarization](https://mlsanity.com/paper/2204.02213/) (2022)
- [OntoGUM: Evaluating Contextualized SOTA Coreference Resolution on 12 More Genres](https://mlsanity.com/paper/2106.00933/) (2021)
- [Lexical Features in Coreference Resolution: To be Used With Caution](https://mlsanity.com/paper/1704.06779/) (2017)
- [Entity-based SpanCopy for Abstractive Summarization to Improve the Factual Consistency](https://mlsanity.com/paper/2209.03479/) (2022)

## Built on by (most cited first, 6 of 6)

- [Major Entity Identification: A Generalizable Alternative to Coreference Resolution](https://mlsanity.com/paper/2406.14654/) (2024)
- [GUM-SAGE: A Novel Dataset and Approach for Graded Entity Salience Prediction](https://mlsanity.com/paper/2504.10792/) (2025)
- [DiscoTrack: A Multilingual LLM Benchmark for Discourse Tracking](https://mlsanity.com/paper/2510.17013/) (2025)
- [Not Worth Mentioning? A Pilot Study on Salient Proposition Annotation](https://mlsanity.com/paper/2603.27358/) (2026)
- [What makes an entity salient in discourse?](https://mlsanity.com/paper/2508.16464/) (2025)
- [The Sum of All Summaries: PROPALIGN for Graded Proposition Salience Evaluation](https://mlsanity.com/paper/2610.19387/) (2026)

## Abstract (arXiv)

As NLP models become increasingly capable of understanding documents in terms of coherent entities rather than strings, obtaining the most salient entities for each document is not only an important end task in itself but also vital for Information Retrieval (IR) and other downstream applications such as controllable summarization. In this paper, we present and evaluate GUMsley, the first entity salience dataset covering all named and non-named salient entities for 12 genres of English text, aligned with entity types, Wikification links and full coreference resolution annotations. We promote a strict definition of salience using human summaries and demonstrate high inter-annotator agreement for salience based on whether a source entity is mentioned in the summary. Our evaluation shows poor performance by pre-trained SOTA summarization models and zero-shot LLM prompting in capturing salient entities in generated summaries. We also show that predicting or providing salient entities to several model architectures enhances performance and helps derive higher-quality summaries by alleviating the entity hallucination problem in existing abstractive summarization.