# RoLargeSum: A Large Dialect-Aware Romanian News Dataset for Summary, Headline, and Keyword Generation

arXiv: https://arxiv.org/abs/2412.11317 · HTML: https://mlsanity.com/paper/2412.11317/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 60 references that are in the map.
- First posted on arXiv 15 Dec 2024.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://mlsanity.com/paper/1911.00359/) (2019)
- [Extractive Summarization as Text Matching](https://mlsanity.com/paper/2004.08795/) (2020)
- [BookSum: A Collection of Datasets for Long-form Narrative Summarization](https://mlsanity.com/paper/2105.08209/) (2021)
- [Ranking Sentences for Extractive Summarization with Reinforcement Learning](https://mlsanity.com/paper/1802.08636/) (2018)
- [Extractive Summarization via ChatGPT for Faithful Summary Generation](https://mlsanity.com/paper/2304.04193/) (2023)
- [BanditSum: Extractive Summarization as a Contextual Bandit](https://mlsanity.com/paper/1809.09672/) (2018)
- [BARThez: a Skilled Pretrained French Sequence-to-Sequence Model](https://mlsanity.com/paper/2010.12321/) (2020)
- [Jointly Extracting and Compressing Documents with Summary State Representations](https://mlsanity.com/paper/1904.02020/) (2019)
- [Introducing RONEC -- the Romanian Named Entity Corpus](https://mlsanity.com/paper/1909.01247/) (2019)
- [SaRoCo: Detecting Satire in a Novel Romanian Corpus of News Articles](https://mlsanity.com/paper/2105.06456/) (2021)
- [RoQLlama: A Lightweight Romanian Adapted Language Model](https://mlsanity.com/paper/2410.04269/) (2024)
- [HistNERo: Historical Named Entity Recognition for the Romanian Language](https://mlsanity.com/paper/2405.00155/) (2024)
- [From News to Summaries: Building a Hungarian Corpus for Extractive and Abstractive Summarization](https://mlsanity.com/paper/2404.03555/) (2024)
- [Adversarial Capsule Networks for Romanian Satire Detection and Sentiment Analysis](https://mlsanity.com/paper/2306.07845/) (2023)

## Built on by (most cited first, 3 of 3)

- [MoRoVoc: A Large Dataset for Geographical Variation Identification of the Spoken Romanian Language](https://mlsanity.com/paper/2509.16781/) (2025)
- [UniBERT: Adversarial Training for Language-Universal Representations](https://mlsanity.com/paper/2503.12608/) (2025)
- [ROMPAR: Morphological Completion and Demographic Unlearning for Romanian-Accented Speech Recognition](https://mlsanity.com/paper/2606.15984/) (2026)

## Abstract (arXiv)

Using supervised automatic summarisation methods requires sufficient corpora that include pairs of documents and their summaries. Similarly to many tasks in natural language processing, most of the datasets available for summarization are in English, posing challenges for developing summarization models in other languages. Thus, in this work, we introduce RoLargeSum, a novel large-scale summarization dataset for the Romanian language crawled from various publicly available news websites from Romania and the Republic of Moldova that were thoroughly cleaned to ensure a high-quality standard. RoLargeSum contains more than 615K news articles, together with their summaries, as well as their headlines, keywords, dialect, and other metadata that we found on the targeted websites. We further evaluated the performance of several BART variants and open-source large language models on RoLargeSum for benchmarking purposes. We manually evaluated the results of the best-performing system to gain insight into the potential pitfalls of this data set and future development.