# Evaluation of Unsupervised Entity and Event Salience Estimation

arXiv: https://arxiv.org/abs/2104.06924 · HTML: https://mlsanity.com/paper/2104.06924/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 24 references that are in the map.
- First posted on arXiv 14 Apr 2021.

## Builds on (most cited first, 4 of 4)

- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Establishing Strong Baselines for the New Decade: Sequence Tagging, Syntactic and Semantic Parsing with BERT](https://mlsanity.com/paper/1908.04943/) (2019)
- [Automatic Event Salience Identification](https://mlsanity.com/paper/1809.00647/) (2018)
- [Towards Better Text Understanding and Retrieval through Kernel Entity Salience Modeling](https://mlsanity.com/paper/1805.01334/) (2018)

## Built on by (most cited first, 3 of 3)

- [How Can Graph Neural Networks Help Document Retrieval: A Case Study on CORD19 with Concept Map Generation](https://mlsanity.com/paper/2201.04672/) (2022)
- [Weakly Supervised Concept Map Generation through Task-Guided Graph Translation](https://mlsanity.com/paper/2110.15720/) (2021)
- [Scalable Detection of Salient Entities in News Articles](https://mlsanity.com/paper/2405.20461/) (2024)

## Abstract (arXiv)

Salience Estimation aims to predict term importance in documents. Due to few existing human-annotated datasets and the subjective notion of salience, previous studies typically generate pseudo-ground truth for evaluation. However, our investigation reveals that the evaluation protocol proposed by prior work is difficult to replicate, thus leading to few follow-up studies existing. Moreover, the evaluation process is problematic: the entity linking tool used for entity matching is very noisy, while the ignorance of event argument for event evaluation leads to boosted performance. In this work, we propose a light yet practical entity and event salience estimation evaluation protocol, which incorporates the more reliable syntactic dependency parser. Furthermore, we conduct a comprehensive analysis among popular entity and event definition standards, and present our own definition for the Salience Estimation task to reduce noise during the pseudo-ground truth generation process. Furthermore, we construct dependency-based heterogeneous graphs to capture the interactions of entities and events. The empirical results show that both baseline methods and the novel GNN method utilizing the heterogeneous graph consistently outperform the previous SOTA model in all proposed metrics.