# Transformer-based Arabic Dialect Identification

arXiv: https://arxiv.org/abs/2011.00699 · HTML: https://mlsanity.com/paper/2011.00699/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2020.

## Built on by (most cited first, 5 of 5)

- [On the Robustness of Arabic Speech Dialect Identification](https://mlsanity.com/paper/2306.03789/) (2023)
- [A Parameter-Efficient Learning Approach to Arabic Dialect Identification with Pre-Trained General-Purpose Speech Model](https://mlsanity.com/paper/2305.11244/) (2023)
- [Two-stage Pipeline for Multilingual Dialect Detection](https://mlsanity.com/paper/2303.03487/) (2023)
- [Unsupervised Out-of-Distribution Dialect Detection with Mahalanobis Distance](https://mlsanity.com/paper/2308.04886/) (2023)
- [Voice Conversion Improves Cross-Domain Robustness for Spoken Arabic Dialect Identification](https://mlsanity.com/paper/2505.24713/) (2025)

## Abstract (arXiv)

This paper presents a dialect identification (DID) system based on the transformer neural network architecture. The conventional convolutional neural network (CNN)-based systems use the shorter receptive fields. We believe that long range information is equally important for language and DID, and self-attention mechanism in transformer captures the long range dependencies. In addition, to reduce the computational complexity, self-attention with downsampling is used to process the acoustic features. This process extracts sparse, yet informative features. Our experimental results show that transformer outperforms CNN-based networks on the Arabic dialect identification (ADI) dataset. We also report that the score-level fusion of CNN and transformer-based systems obtains an overall accuracy of 86.29% on the ADI17 database.