# Ara-Best-RQ: Multi Dialectal Arabic SSL

arXiv: https://arxiv.org/abs/2603.21900 · HTML: https://mlsanity.com/paper/2603.21900/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 23 Mar 2026.

## Builds on (most cited first, 17 of 17)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages](https://mlsanity.com/paper/2303.01037/) (2023)
- [Seamless: Multilingual Expressive and Streaming Speech Translation](https://mlsanity.com/paper/2312.05187/) (2023)
- [SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing](https://mlsanity.com/paper/2110.07205/) (2021)
- [Self-supervised Learning with Random-projection Quantizer for Speech Recognition](https://mlsanity.com/paper/2202.01855/) (2022)
- [Open-Source Conversational AI with SpeechBrain 1.0](https://mlsanity.com/paper/2407.00463/) (2024)
- [QASR: QCRI Aljazeera Speech Resource -- A Large Scale Annotated Arabic Speech Corpus](https://mlsanity.com/paper/2106.13000/) (2021)
- [AfriHuBERT: A self-supervised speech representation model for African languages](https://mlsanity.com/paper/2409.20201/) (2024)
- [MediaSpeech: Multilanguage ASR Benchmark and Dataset](https://mlsanity.com/paper/2103.16193/) (2021)
- [ArTST: Arabic Text and Speech Transformer](https://mlsanity.com/paper/2310.16621/) (2023)
- [Mixat: A Data Set of Bilingual Emirati-English Speech](https://mlsanity.com/paper/2405.02578/) (2024)
- [Performance Analysis of Speech Encoders for Low-Resource SLU and ASR in Tunisian Dialect](https://mlsanity.com/paper/2407.04533/) (2024)
- [LinTO Audio and Textual Datasets to Train and Evaluate Automatic Speech Recognition in Tunisian Arabic Dialect](https://mlsanity.com/paper/2504.02604/) (2025)
- [Pantagruel: Unified Self-Supervised Encoders for French Text and Speech](https://mlsanity.com/paper/2601.05911/) (2026)

## Built on by (most cited first, 2 of 2)

- [NADI 2026: The Second Multidialectal Arabic Speech Processing Shared Task](https://mlsanity.com/paper/2609.27086/) (2026)
- [Nufo: A Benchmark for Ewe Automatic Speech Recognition](https://mlsanity.com/paper/2610.17278/) (2026)

## Abstract (arXiv)

We present Ara-BEST-RQ, a family of self-supervised learning (SSL) models specifically designed for multi-dialectal Arabic speech processing. Leveraging 5,640 hours of crawled Creative Commons speech and combining it with publicly available datasets, we pre-train conformer-based BEST-RQ models up to 600M parameters. Our models are evaluated on dialect identification (DID) and automatic speech recognition (ASR) tasks, achieving state-of-the-art performance on the former while using fewer parameters than competing models. We demonstrate that family-targeted pre-training on Arabic dialects significantly improves downstream performance compared to multilingual or monolingual models trained on non-Arabic data. All models, code, and pre-processed datasets will be publicly released to support reproducibility and further research in Arabic speech technologies.