# Multi-Dialect Arabic BERT for Country-Level Dialect Identification

arXiv: https://arxiv.org/abs/2007.05612 · HTML: https://mlsanity.com/paper/2007.05612/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2020.

## Built on by (most cited first, 15 of 15)

- [The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models](https://mlsanity.com/paper/2103.06678/) (2021)
- [Pre-Training BERT on Arabic Tweets: Practical Considerations](https://mlsanity.com/paper/2102.10684/) (2021)
- [Masader: Metadata Sourcing for Arabic Text and Speech Data Resources](https://mlsanity.com/paper/2110.06744/) (2021)
- [A Survey of Large Language Models for Arabic Language and its Dialects](https://mlsanity.com/paper/2410.20238/) (2024)
- [Finnish Dialect Identification: The Effect of Audio and Text](https://mlsanity.com/paper/2111.03800/) (2021)
- [BERT-based Multi-Task Model for Country and Province Level Modern Standard Arabic and Dialectal Arabic Identification](https://mlsanity.com/paper/2106.12495/) (2021)
- [An open access NLP dataset for Arabic dialects : Data collection, labeling, and model construction](https://mlsanity.com/paper/2102.11000/) (2021)
- [Adapting MARBERT for Improved Arabic Dialect Identification: Submission to the NADI 2021 Shared Task](https://mlsanity.com/paper/2103.01065/) (2021)
- [Similarities between Arabic Dialects: Investigating Geographical Proximity](https://mlsanity.com/paper/2105.04221/) (2021)
- [Bench-Marking And Improving Arabic Automatic Image Captioning Through The Use Of Multi-Task Learning Paradigm](https://mlsanity.com/paper/2202.05474/) (2022)
- [Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Understanding](https://mlsanity.com/paper/2205.10687/) (2022)
- [Revisiting Common Assumptions about Arabic Dialects in NLP](https://mlsanity.com/paper/2505.21816/) (2025)
- [sarcasm detection and quantification in arabic tweets](https://mlsanity.com/paper/2108.01425/) (2021)
- [AlcLaM: Arabic Dialectal Language Model](https://mlsanity.com/paper/2407.13097/) (2024)
- [Exploring Data and Parameter Efficient Strategies for Arabic Dialect Identifications](https://mlsanity.com/paper/2509.13775/) (2025)

## Abstract (arXiv)

Arabic dialect identification is a complex problem for a number of inherent properties of the language itself. In this paper, we present the experiments conducted, and the models developed by our competing team, Mawdoo3 AI, along the way to achieving our winning solution to subtask 1 of the Nuanced Arabic Dialect Identification (NADI) shared task. The dialect identification subtask provides 21,000 country-level labeled tweets covering all 21 Arab countries. An unlabeled corpus of 10M tweets from the same domain is also presented by the competition organizers for optional use. Our winning solution itself came in the form of an ensemble of different training iterations of our pre-trained BERT model, which achieved a micro-averaged F1-score of 26.78% on the subtask at hand. We publicly release the pre-trained language model component of our winning solution under the name of Multi-dialect-Arabic-BERT model, for any interested researcher out there.