# Beyond Arabic: Software for Perso-Arabic Script Manipulation

arXiv: https://arxiv.org/abs/2301.11406 · HTML: https://mlsanity.com/paper/2301.11406/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 41 references that are in the map.
- First posted on arXiv 26 Jan 2023.

## Builds on (most cited first, 5 of 5)

- [Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets](https://mlsanity.com/paper/2103.12028/) (2021)
- [Building Machine Translation Systems for the Next Thousand Languages](https://mlsanity.com/paper/2205.03983/) (2022)
- [Towards Machine Translation for the Kurdish Language](https://mlsanity.com/paper/2010.06041/) (2020)
- [Graphemic Normalization of the Perso-Arabic Script](https://mlsanity.com/paper/2210.12273/) (2022)
- [Urdu Morphology, Orthography and Lexicon Extraction](https://mlsanity.com/paper/2204.03071/) (2022)

## Built on by (most cited first, 2 of 2)

- [XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages](https://mlsanity.com/paper/2305.11938/) (2023)
- [Script Normalization for Unconventional Writing of Under-Resourced Languages in Bilingual Communities](https://mlsanity.com/paper/2305.16407/) (2023)

## Abstract (arXiv)

This paper presents an open-source software library that provides a set of finite-state transducer (FST) components and corresponding utilities for manipulating the writing systems of languages that use the Perso-Arabic script. The operations include various levels of script normalization, including visual invariance-preserving operations that subsume and go beyond the standard Unicode normalization forms, as well as transformations that modify the visual appearance of characters in accordance with the regional orthographies for eleven contemporary languages from diverse language families. The library also provides simple FST-based romanization and transliteration. We additionally attempt to formalize the typology of Perso-Arabic characters by providing one-to-many mappings from Unicode code points to the languages that use them. While our work focuses on the Arabic script diaspora rather than Arabic itself, this approach could be adopted for any language that uses the Arabic script, thus providing a unified framework for treating a script family used by close to a billion people.