# BERT-based Multi-Task Model for Country and Province Level Modern Standard Arabic and Dialectal Arabic Identification

arXiv: https://arxiv.org/abs/2106.12495 · HTML: https://mlsanity.com/paper/2106.12495/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 23 Jun 2021.

## Builds on (most cited first, 7 of 7)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic](https://mlsanity.com/paper/2101.01785/) (2020)
- [NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2010.11334/) (2020)
- [NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2103.08466/) (2021)
- [Multi-Dialect Arabic BERT for Country-Level Dialect Identification](https://mlsanity.com/paper/2007.05612/) (2020)

## Built on by (most cited first, 7 of 7)

- [Arabic Text Diacritization In The Age Of Transfer Learning: Token Classification Is All You Need](https://mlsanity.com/paper/2401.04848/) (2024)
- [UM6P-CS at SemEval-2022 Task 11: Enhancing Multilingual and Code-Mixed Complex Named Entity Recognition via Pseudo Labels using Multilingual Transformer](https://mlsanity.com/paper/2204.13515/) (2022)
- [Revisiting Common Assumptions about Arabic Dialects in NLP](https://mlsanity.com/paper/2505.21816/) (2025)
- [Deep Multi-Task Models for Misogyny Identification and Categorization on Arabic Social Media](https://mlsanity.com/paper/2206.08407/) (2022)
- [DarijaBanking: A New Resource for Overcoming Language Barriers in Banking Intent Detection for Moroccan Arabic Speakers](https://mlsanity.com/paper/2405.16482/) (2024)
- [Overview of the Arabic Sentiment Analysis 2021 Competition at KAUST](https://mlsanity.com/paper/2109.14456/) (2021)
- [CS-UM6P at SemEval-2022 Task 6: Transformer-based Models for Intended Sarcasm Detection in English and Arabic](https://mlsanity.com/paper/2206.08415/) (2022)

## Abstract (arXiv)

Dialect and standard language identification are crucial tasks for many Arabic natural language processing applications. In this paper, we present our deep learning-based system, submitted to the second NADI shared task for country-level and province-level identification of Modern Standard Arabic (MSA) and Dialectal Arabic (DA). The system is based on an end-to-end deep Multi-Task Learning (MTL) model to tackle both country-level and province-level MSA/DA identification. The latter MTL model consists of a shared Bidirectional Encoder Representation Transformers (BERT) encoder, two task-specific attention layers, and two classifiers. Our key idea is to leverage both the task-discriminative and the inter-task shared features for country and province MSA/DA identification. The obtained results show that our MTL model outperforms single-task models on most subtasks.