# Maastricht University at AMIYA: Adapting LLMs for Dialectal Arabic using Fine-tuning and MBR Decoding

arXiv: https://arxiv.org/abs/2602.09703 · HTML: https://mlsanity.com/paper/2602.09703/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 19 references that are in the map.
- First posted on arXiv 10 Feb 2026.

## Builds on (most cited first, 7 of 7)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Simple, Scalable Adaptation for Neural Machine Translation](https://mlsanity.com/paper/1909.08478/) (2019)
- [ALDi: Quantifying the Arabic Level of Dialectness of Text](https://mlsanity.com/paper/2310.13747/) (2023)
- [Towards Responsible Natural Language Annotation for the Varieties of Arabic](https://mlsanity.com/paper/2203.09597/) (2022)
- [mbrs: A Library for Minimum Bayes Risk Decoding](https://mlsanity.com/paper/2408.04167/) (2024)
- [AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabic](https://mlsanity.com/paper/2412.04193/) (2024)

## Abstract (arXiv)

Large Language Models (LLMs) are becoming increasingly multilingual, supporting hundreds of languages, especially high resource ones. Unfortunately, Dialect variations are still underrepresented due to limited data and linguistic variation. In this work, we adapt a pre-trained LLM to improve dialectal performance. Specifically, we use Low Rank Adaptation (LoRA) fine-tuning on monolingual and English Dialect parallel data, adapter merging and dialect-aware MBR decoding to improve dialectal fidelity generation and translation. Experiments on Syrian, Moroccan, and Saudi Arabic show that merging and MBR improve dialectal fidelity while preserving semantic accuracy. This combination provides a compact and effective framework for robust dialectal Arabic generation.