# Morphosyntactic Tagging with Pre-trained Language Models for Arabic and its Dialects

arXiv: https://arxiv.org/abs/2110.06852 · HTML: https://mlsanity.com/paper/2110.06852/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 52 references that are in the map.
- First posted on arXiv 13 Oct 2021.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models](https://mlsanity.com/paper/2103.06678/) (2021)
- [The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection](https://mlsanity.com/paper/1910.11493/) (2019)
- [Pre-Training BERT on Arabic Tweets: Practical Considerations](https://mlsanity.com/paper/2102.10684/) (2021)
- [Arabizi Detection and Conversion to Arabic](https://mlsanity.com/paper/1306.6755/) (2013)
- [Joint Diacritization, Lemmatization, Normalization, and Fine-Grained Morphological Tagging](https://mlsanity.com/paper/1910.02267/) (2019)
- [LDC Arabic Treebanks and Associated Corpora: Data Divisions Manual](https://mlsanity.com/paper/1309.5652/) (2013)
- [Adversarial Multitask Learning for Joint Multi-Feature and Multi-Dialect Morphological Modeling](https://mlsanity.com/paper/1910.12702/) (2019)
- [Self-Training Pre-Trained Language Models for Zero- and Few-Shot Multi-Dialectal Arabic Sequence Labeling](https://mlsanity.com/paper/2101.04758/) (2021)

## Built on by (most cited first, 14 of 14)

- [Natural Language Processing for Dialects of a Language: A Survey](https://mlsanity.com/paper/2401.05632/) (2024)
- [Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigation](https://mlsanity.com/paper/2305.14734/) (2023)
- [Camelira: An Arabic Multi-Dialect Morphological Disambiguator](https://mlsanity.com/paper/2211.16807/) (2022)
- [ZAEBUC-Spoken: A Multilingual Multidialectal Arabic-English Speech Corpus](https://mlsanity.com/paper/2403.18182/) (2024)
- [The SAMER Arabic Text Simplification Corpus](https://mlsanity.com/paper/2404.18615/) (2024)
- [Arabic Diacritics in the Wild: Exploiting Opportunities for Improved Diacritization](https://mlsanity.com/paper/2406.05760/) (2024)
- [Strategies for Arabic Readability Modeling](https://mlsanity.com/paper/2407.03032/) (2024)
- [Transformers on Multilingual Clause-Level Morphology](https://mlsanity.com/paper/2211.01736/) (2022)
- [Computational Morphology and Lexicography Modeling of Modern Standard Arabic Nominals](https://mlsanity.com/paper/2402.00385/) (2024)
- [A Tale of Two Scripts: Transliteration and Post-Correction for Judeo-Arabic](https://mlsanity.com/paper/2507.04746/) (2025)
- [Lemmatization as a Classification Task: Results from Arabic across Multiple Genres](https://mlsanity.com/paper/2506.18399/) (2025)
- [Data Augmentation for Maltese NLP using Transliterated and Machine Translated Arabic Data](https://mlsanity.com/paper/2509.12853/) (2025)
- [Arabic Morphosyntactic Tagging and Dependency Parsing with Large Language Models](https://mlsanity.com/paper/2603.16718/) (2026)
- [YallaMorph: A Benchmark for Evaluating Arabic Morphological Generation in Large Language Models](https://mlsanity.com/paper/2609.10153/) (2026)

## Abstract (arXiv)

We present state-of-the-art results on morphosyntactic tagging across different varieties of Arabic using fine-tuned pre-trained transformer language models. Our models consistently outperform existing systems in Modern Standard Arabic and all the Arabic dialects we study, achieving 2.6% absolute improvement over the previous state-of-the-art in Modern Standard Arabic, 2.8% in Gulf, 1.6% in Egyptian, and 8.3% in Levantine. We explore different training setups for fine-tuning pre-trained transformer language models, including training data size, the use of external linguistic resources, and the use of annotated data from other dialects in a low-resource scenario. Our results show that strategic fine-tuning using datasets from other high-resource dialects is beneficial for a low-resource dialect. Additionally, we show that high-quality morphological analyzers as external linguistic resources are beneficial especially in low-resource settings.