# An Evaluation Framework for Mapping News Headlines to Event Classes in a Knowledge Graph

arXiv: https://arxiv.org/abs/2312.02334 · HTML: https://mlsanity.com/paper/2312.02334/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 4 Dec 2023.

## Builds on (most cited first, 8 of 8)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://mlsanity.com/paper/1909.00161/) (2019)
- [Falcon 2.0: An Entity and Relation Linking Tool over Wikidata](https://mlsanity.com/paper/1912.11270/) (2019)
- [OpenTapioca: Lightweight Entity Linking for Wikidata](https://mlsanity.com/paper/1904.09131/) (2019)
- [Event Linking: Grounding Event Mentions to Wikipedia](https://mlsanity.com/paper/2112.07888/) (2021)
- [EMBERS at 4 years: Experiences operating an Open Source Indicators Forecasting System](https://mlsanity.com/paper/1604.00033/) (2016)

## Built on by (most cited first, 1 of 1)

- [ACCESS : A Benchmark for Abstract Causal Event Discovery and Reasoning](https://mlsanity.com/paper/2502.08148/) (2025)

## Abstract (arXiv)

Mapping ongoing news headlines to event-related classes in a rich knowledge base can be an important component in a knowledge-based event analysis and forecasting solution. In this paper, we present a methodology for creating a benchmark dataset of news headlines mapped to event classes in Wikidata, and resources for the evaluation of methods that perform the mapping. We use the dataset to study two classes of unsupervised methods for this task: 1) adaptations of classic entity linking methods, and 2) methods that treat the problem as a zero-shot text classification problem. For the first approach, we evaluate off-the-shelf entity linking systems. For the second approach, we explore a) pre-trained natural language inference (NLI) models, and b) pre-trained large generative language models. We present the results of our evaluation, lessons learned, and directions for future work. The dataset and scripts for evaluation are made publicly available.