# Can Multilingual Language Models Transfer to an Unseen Dialect? A Case Study on North African Arabizi

arXiv: https://arxiv.org/abs/2005.00318 · HTML: https://mlsanity.com/paper/2005.00318/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 May 2020.

## Built on by (most cited first, 13 of 13)

- [The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants](https://mlsanity.com/paper/2308.16884/) (2023)
- [A Primer on Pretrained Multilingual Language Models](https://mlsanity.com/paper/2107.00676/) (2021)
- [MaLA-500: Massive Language Adaptation of Large Language Models](https://mlsanity.com/paper/2401.13303/) (2024)
- [Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots](https://mlsanity.com/paper/2103.09593/) (2021)
- [Improving Zero-shot Cross-lingual Transfer between Closely Related Languages by injecting Character-level Noise](https://mlsanity.com/paper/2109.06772/) (2021)
- [Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?](https://mlsanity.com/paper/2110.13658/) (2021)
- [Analyzing Gender Representation in Multilingual Models](https://mlsanity.com/paper/2204.09168/) (2022)
- [The interplay between language similarity and script on a novel multi-layer Algerian dialect corpus](https://mlsanity.com/paper/2105.07400/) (2021)
- [A Survey of Code-switched Arabic NLP: Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2501.13419/) (2025)
- [Enriching the NArabizi Treebank: A Multifaceted Approach to Supporting an Under-Resourced Language](https://mlsanity.com/paper/2306.14866/) (2023)
- [Transfer Learning for Mining Feature Requests and Bug Reports from Tweets and App Store Reviews](https://mlsanity.com/paper/2108.00663/) (2021)
- [Low-Resource Clickbait Spoiling for Indonesian via Question Answering](https://mlsanity.com/paper/2310.08085/) (2023)
- [Adapting Multilingual Models to Code-Mixed Tasks via Model Merging](https://mlsanity.com/paper/2510.19782/) (2025)

## Abstract (arXiv)

Building natural language processing systems for non standardized and low resource languages is a difficult challenge. The recent success of large-scale multilingual pretrained language models provides new modeling tools to tackle this. In this work, we study the ability of multilingual language models to process an unseen dialect. We take user generated North-African Arabic as our case study, a resource-poor dialectal variety of Arabic with frequent code-mixing with French and written in Arabizi, a non-standardized transliteration of Arabic to Latin script. Focusing on two tasks, part-of-speech tagging and dependency parsing, we show in zero-shot and unsupervised adaptation scenarios that multilingual language models are able to transfer to such an unseen dialect, specifically in two extreme cases: (i) across scripts, using Modern Standard Arabic as a source language, and (ii) from a distantly related language, unseen during pretraining, namely Maltese. Our results constitute the first successful transfer experiments on this dialect, paving thus the way for the development of an NLP ecosystem for resource-scarce, non-standardized and highly variable vernacular languages.