Benchmarking Abstractive Summarisation: A Dataset of Human-authored Summaries of Norwegian News Articles
Not on the map. Why?
Abstract (arXiv)
We introduce a dataset of high-quality human-authored summaries of news articles in Norwegian. The dataset is intended for benchmarking the abstractive summarisation capabilities of generative language models. Each document in the dataset is provided with three different candidate gold-standard summaries written by native Norwegian speakers, and all summaries are provided in both of the written variants of Norwegian -- Bokmaal and Nynorsk. The paper describes details on the data creation effort as well as an evaluation of existing open LLMs for Norwegian on the dataset. We also provide insights from a manual human evaluation, comparing human-authored to model-generated summaries. Our results indicate that the dataset provides a challenging LLM benchmark for Norwegian summarisation capabilities
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 12 of its 21 references that are in the map. First posted on arXiv 13 Jan 2025.
Builds on
12 of 21 references.
References · 12
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
- 2015Teaching Machines to Read and Comprehendcited by 1,039
- 2020SummEval: Re-evaluating Summarization Evaluationcited by 666
- 2019Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Modelcited by 431
- 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 394
- 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 154
- 2024Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvementscited by 82
- 2023How are Prompts Different in Terms of Sensitivity?cited by 34
- 2019NorNE: Annotating Named Entities for Norwegiancited by 18
- 2024Small Languages, Big Models: A Study of Continual Training on Languages of Norwaycited by 12
- 2022The Glass Ceiling of Automatic Evaluation in Natural Language Generationcited by 10
Built on by
Citing papers per year: 2025: 2.
2 papers.