Event-based evaluation of abstractive news summarization
Not on the map. Why?
Abstract (arXiv)
An abstractive summary of a news article contains its most important information in a condensed version. The evaluation of automatically generated summaries by generative language models relies heavily on human-authored summaries as gold references, by calculating overlapping units or similarity scores. News articles report events, and ideally so should the summaries. In this work, we propose to evaluate the quality of abstractive summaries by calculating overlapping events between generated summaries, reference summaries, and the original news articles. We experiment on a richly annotated Norwegian dataset comprising both events annotations and summaries authored by expert human annotators. Our approach provides more insight into the event information contained in the summaries.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 6 of its 14 references that are in the map. First posted on arXiv 1 Jul 2025.
Builds on
6 of 14 references.
References · 6
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
- 2022Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluationcited by 142
- 2021Finding a Balanced Degree of Automation for Summary Evaluationcited by 43
- 2021Cross-Task Instance Representation Interactions and Label Dependencies for Joint Information Extraction with Graph Convolutional Networkscited by 22
- 2025Benchmarking Abstractive Summarisation: A Dataset of Human-authored Summaries of Norwegian News Articlescited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.