# YallaMorph: A Benchmark for Evaluating Arabic Morphological Generation in Large Language Models

arXiv: https://arxiv.org/abs/2609.10153 · HTML: https://mlsanity.com/paper/2609.10153/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 32 references that are in the map.
- First posted on arXiv 9 Sep 2026.

## Builds on (most cited first, 10 of 10)

- [ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic](https://mlsanity.com/paper/2402.12840/) (2024)
- [AraT5: Text-to-Text Transformers for Arabic Language Generation](https://mlsanity.com/paper/2109.12068/) (2021)
- [The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection](https://mlsanity.com/paper/1910.11493/) (2019)
- [The CoNLL--SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection](https://mlsanity.com/paper/1810.07125/) (2018)
- [CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages](https://mlsanity.com/paper/1706.09031/) (2017)
- [SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection](https://mlsanity.com/paper/2006.11572/) (2020)
- [Evaluating Morphological Compositional Generalization in Large Language Models](https://mlsanity.com/paper/2410.12656/) (2024)
- [Morphosyntactic Tagging with Pre-trained Language Models for Arabic and its Dialects](https://mlsanity.com/paper/2110.06852/) (2021)
- [IMPACT: Inflectional Morphology Probes Across Complex Typologies](https://mlsanity.com/paper/2506.23929/) (2025)
- [Morphemes Without Borders: Evaluating Root-Pattern Morphology in Arabic Tokenizers and LLMs](https://mlsanity.com/paper/2603.15773/) (2026)

## Abstract (arXiv)

Arabic morphology remains challenging for large language models, since fluent generation does not guarantee accurate morphosyntactic control. Existing Arabic evaluations mainly target downstream tasks and do not directly test controlled morphological generation from explicit lexical and feature-based input. We introduce YallaMorph, a large-scale benchmark for Arabic morphological generation covering verbs, nouns, adjectives, their cliticized forms, and invalid configurations. We evaluate multilingual and Arabic-oriented LLMs under diacritized and undiacritized settings over 600K benchmark entries. Results show that Arabic morphological generation remains difficult, especially for cliticized, unseen, and morphologically rare forms.