# Text normalization for low-resource languages: the case of Ligurian

arXiv: https://arxiv.org/abs/2206.07861 · HTML: https://mlsanity.com/paper/2206.07861/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 16 Jun 2022.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages](https://mlsanity.com/paper/2010.02353/) (2020)
- [Applying the Transformer to Character-level Transduction](https://mlsanity.com/paper/2005.10213/) (2020)
- [A Large-Scale Comparison of Historical Text Normalization Systems](https://mlsanity.com/paper/1904.02036/) (2019)
- [Exact Hard Monotonic Attention for Character-Level Transduction](https://mlsanity.com/paper/1905.06319/) (2019)
- [Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?](https://mlsanity.com/paper/2110.13658/) (2021)
- [An Evaluation of Neural Machine Translation Models on Historical Spelling Normalization](https://mlsanity.com/paper/1806.05210/) (2018)
- [Text Normalization for Low-Resource Languages of Africa](https://mlsanity.com/paper/2103.15845/) (2021)

## Built on by (most cited first, 4 of 4)

- [Script Normalization for Unconventional Writing of Under-Resourced Languages in Bilingual Communities](https://mlsanity.com/paper/2305.16407/) (2023)
- [MERLIN: Multi-Stage Curriculum Alignment for Multilingual Encoder-LLM Integration in Cross-Lingual Reasoning](https://mlsanity.com/paper/2509.08105/) (2025)
- [Language Varieties of Italy: Technology Challenges and Opportunities](https://mlsanity.com/paper/2209.09757/) (2022)
- [Automatic Correction of Writing Anomalies in Hausa Texts](https://mlsanity.com/paper/2506.03820/) (2025)

## Abstract (arXiv)

Text normalization is a crucial technology for low-resource languages which lack rigid spelling conventions or that have undergone multiple spelling reforms. Low-resource text normalization has so far relied upon hand-crafted rules, which are perceived to be more data efficient than neural methods. In this paper we examine the case of text normalization for Ligurian, an endangered Romance language. We collect 4,394 Ligurian sentences paired with their normalized versions, as well as the first open source monolingual corpus for Ligurian. We show that, in spite of the small amounts of data available, a compact transformer-based model can be trained to achieve very low error rates by the use of backtranslation and appropriate tokenization.