# HeSum: a Novel Dataset for Abstractive Text Summarization in Hebrew

arXiv: https://arxiv.org/abs/2406.03897 · HTML: https://mlsanity.com/paper/2406.03897/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 6 Jun 2024.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Language Models are Multilingual Chain-of-Thought Reasoners](https://mlsanity.com/paper/2210.03057/) (2022)
- [XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages](https://mlsanity.com/paper/2106.13822/) (2021)
- [MEGA: Multilingual Evaluation of Generative AI](https://mlsanity.com/paper/2303.12528/) (2023)
- [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](https://mlsanity.com/paper/2304.05613/) (2023)
- [Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challenges](https://mlsanity.com/paper/2311.03287/) (2023)
- [Hallucinations in Large Multilingual Translation Models](https://mlsanity.com/paper/2303.16104/) (2023)
- [Superbizarre Is Not Superb: Derivational Morphology Improves BERT's Interpretation of Complex Words](https://mlsanity.com/paper/2101.00403/) (2021)
- [From SPMRL to NMRL: What Did We Learn (and Unlearn) in a Decade of Parsing Morphologically-Rich Languages (MRLs)?](https://mlsanity.com/paper/2005.01330/) (2020)
- [Neural Modeling for Named Entities and Morphology (NEMO^2)](https://mlsanity.com/paper/2007.15620/) (2020)
- [DictaBERT: A State-of-the-Art BERT Suite for Modern Hebrew](https://mlsanity.com/paper/2308.16687/) (2023)
- [What's Wrong with Hebrew NLP? And How to Make it Right](https://mlsanity.com/paper/1908.05453/) (2019)
- [How well do you know your summarization datasets?](https://mlsanity.com/paper/2106.11388/) (2021)
- [ParaShoot: A Hebrew Question Answering Dataset](https://mlsanity.com/paper/2109.11314/) (2021)
- [mLongT5: A Multilingual and Efficient Text-To-Text Transformer for Longer Sequences](https://mlsanity.com/paper/2305.11129/) (2023)
- [CoheSentia: A Novel Benchmark of Incremental versus Holistic Assessment of Coherence in Generated Texts](https://mlsanity.com/paper/2310.16329/) (2023)
- [Explicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew](https://mlsanity.com/paper/2311.00658/) (2023)
- [HeGeL: A Novel Dataset for Geo-Location from Hebrew Text](https://mlsanity.com/paper/2307.00509/) (2023)

## Built on by (most cited first, 6 of 6)

- [Multilingual Performance Biases of Large Language Models in Education](https://mlsanity.com/paper/2504.17720/) (2025)
- [The State and Fate of Summarization Datasets: A Survey](https://mlsanity.com/paper/2411.04585/) (2024)
- [Beyond N-Grams: Rethinking Evaluation Metrics and Strategies for Multilingual Abstractive Summarization](https://mlsanity.com/paper/2507.08342/) (2025)
- [M3FinMeeting: A Multilingual, Multi-Sector, and Multi-Task Financial Meeting Understanding Evaluation Dataset](https://mlsanity.com/paper/2506.02510/) (2025)
- [Leveraging Digitized Newspapers to Collect Summarization Data in Low-Resource Languages](https://mlsanity.com/paper/2511.14598/) (2025)
- [SteerEval: Inference-time Interventions Strengthen Multilingual Generalization in Neural Summarization Metrics](https://mlsanity.com/paper/2601.15809/) (2026)

## Abstract (arXiv)

While large language models (LLMs) excel in various natural language tasks in English, their performance in lower-resourced languages like Hebrew, especially for generative tasks such as abstractive summarization, remains unclear. The high morphological richness in Hebrew adds further challenges due to the ambiguity in sentence comprehension and the complexities in meaning construction. In this paper, we address this resource and evaluation gap by introducing HeSum, a novel benchmark specifically designed for abstractive text summarization in Modern Hebrew. HeSum consists of 10,000 article-summary pairs sourced from Hebrew news websites written by professionals. Linguistic analysis confirms HeSum's high abstractness and unique morphological challenges. We show that HeSum presents distinct difficulties for contemporary state-of-the-art LLMs, establishing it as a valuable testbed for generative language technology in Hebrew, and MRLs generative challenges in general.