# ArTST: Arabic Text and Speech Transformer

arXiv: https://arxiv.org/abs/2310.16621 · HTML: https://mlsanity.com/paper/2310.16621/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 34 references that are in the map.
- First posted on arXiv 25 Oct 2023.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [Scaling Speech Technology to 1,000+ Languages](https://mlsanity.com/paper/2305.13516/) (2023)
- [Revisiting Neural Scaling Laws in Language and Vision](https://mlsanity.com/paper/2209.06640/) (2022)
- [Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Model](https://mlsanity.com/paper/1909.05330/) (2019)
- [Multilingual Speech Recognition With A Single End-To-End Model](https://mlsanity.com/paper/1711.01694/) (2017)
- [AraT5: Text-to-Text Transformers for Arabic Language Generation](https://mlsanity.com/paper/2109.12068/) (2021)
- [QASR: QCRI Aljazeera Speech Resource -- A Large Scale Annotated Arabic Speech Corpus](https://mlsanity.com/paper/2106.13000/) (2021)
- [ClArTTS: An Open-Source Classical Arabic Text-to-Speech Corpus](https://mlsanity.com/paper/2303.00069/) (2023)
- [Heterogeneous Target Speech Separation](https://mlsanity.com/paper/2204.03594/) (2022)
- [On the Robustness of Arabic Speech Dialect Identification](https://mlsanity.com/paper/2306.03789/) (2023)
- [Multilingual Training and Cross-lingual Adaptation on CTC-based Acoustic Model](https://mlsanity.com/paper/1711.10025/) (2017)
- [Supervised Acoustic Embeddings And Their Transferability Across Languages](https://mlsanity.com/paper/2301.01020/) (2023)

## Built on by (most cited first, 14 of 14)

- [Mixat: A Data Set of Bilingual Emirati-English Speech](https://mlsanity.com/paper/2405.02578/) (2024)
- [Performance Analysis of Speech Encoders for Low-Resource SLU and ASR in Tunisian Dialect](https://mlsanity.com/paper/2407.04533/) (2024)
- [LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLM](https://mlsanity.com/paper/2503.04724/) (2025)
- [NADI 2025: The First Multidialectal Arabic Speech Processing Shared Task](https://mlsanity.com/paper/2509.02038/) (2025)
- [To Distill or Not to Distill? On the Robustness of Robust Knowledge Distillation](https://mlsanity.com/paper/2406.04512/) (2024)
- [Voice of a Continent: Mapping Africa's Speech Technology Frontier](https://mlsanity.com/paper/2505.18436/) (2025)
- [ArVoice: A Multi-Speaker Dataset for Arabic Speech Synthesis](https://mlsanity.com/paper/2505.20506/) (2025)
- [Towards Zero-Shot Text-To-Speech for Arabic Dialects](https://mlsanity.com/paper/2406.16751/) (2024)
- [Habibi: Laying the Open-Source Foundation of Unified-Dialectal Arabic Speech Synthesis](https://mlsanity.com/paper/2601.13802/) (2026)
- [Ara-Best-RQ: Multi Dialectal Arabic SSL](https://mlsanity.com/paper/2603.21900/) (2026)
- [Open Automatic Speech Recognition Models for Classical and Modern Standard Arabic](https://mlsanity.com/paper/2507.13977/) (2025)
- [ArFake: A Robust Framework for Multi-Dialect Arabic Speech Spoofing Detection Benchmark](https://mlsanity.com/paper/2509.22808/) (2025)
- [NADI 2026: The Second Multidialectal Arabic Speech Processing Shared Task](https://mlsanity.com/paper/2609.27086/) (2026)
- [CARDAMOM: A Micro-Dialectal Arabic Speech Dataset for ASR](https://mlsanity.com/paper/2609.34481/) (2026)

## Abstract (arXiv)

We present ArTST, a pre-trained Arabic text and speech transformer for supporting open-source speech technologies for the Arabic language. The model architecture follows the unified-modal framework, SpeechT5, that was recently released for English, and is focused on Modern Standard Arabic (MSA), with plans to extend the model for dialectal and code-switched Arabic in future editions. We pre-trained the model from scratch on MSA speech and text data, and fine-tuned it for the following tasks: Automatic Speech Recognition (ASR), Text-To-Speech synthesis (TTS), and spoken dialect identification. In our experiments comparing ArTST with SpeechT5, as well as with previously reported results in these tasks, ArTST performs on a par with or exceeding the current state-of-the-art in all three tasks. Moreover, we find that our pre-training is conducive for generalization, which is particularly evident in the low-resource TTS task. The pre-trained model as well as the fine-tuned ASR and TTS models are released for research use.