Text normalization for low-resource languages: the case of Ligurian
Not on the map. Why?
Abstract (arXiv)
Text normalization is a crucial technology for low-resource languages which lack rigid spelling conventions or that have undergone multiple spelling reforms. Low-resource text normalization has so far relied upon hand-crafted rules, which are perceived to be more data efficient than neural methods. In this paper we examine the case of text normalization for Ligurian, an endangered Romance language. We collect 4,394 Ligurian sentences paired with their normalized versions, as well as the first open source monolingual corpus for Ligurian. We show that, in spite of the small amounts of data available, a compact transformer-based model can be trained to achieve very low error rates by the use of backtranslation and appropriate tokenization.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 12 of its 35 references that are in the map. First posted on arXiv 16 Jun 2022.
Builds on
12 of 35 references.
References · 12
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
- 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
- 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
- 2020Participatory Research for Low-resourced Machine Translation: A Case Study in African Languagescited by 159
- 2020Applying the Transformer to Character-level Transductioncited by 43
- 2019A Large-Scale Comparison of Historical Text Normalization Systemscited by 25
- 2019Exact Hard Monotonic Attention for Character-Level Transductioncited by 22
- 2021Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?cited by 11
- 2018An Evaluation of Neural Machine Translation Models on Historical Spelling Normalizationcited by 5
- 2021Text Normalization for Low-Resource Languages of Africacited by 2
Built on by
Citing papers per year: 2022: 1; 2023: 1; 2024: 0; 2025: 2.
4 papers.
- 2023Script Normalization for Unconventional Writing of Under-Resourced Languages in Bilingual Communitiescited by 2
- 2025MERLIN: Multi-Stage Curriculum Alignment for Multilingual Encoder-LLM Integration in Cross-Lingual Reasoningcited by 2
- 2022Language Varieties of Italy: Technology Challenges and Opportunitiescited by 1
- 2025Automatic Correction of Writing Anomalies in Hausa Textscited by 0
- 2025MERLIN: Multi-Stage Curriculum Alignment for Multilingual Encoder-LLM Integration in Cross-Lingual Reasoning9 Sep 2025
- 2025Automatic Correction of Writing Anomalies in Hausa Texts4 Jun 2025
- 2023Script Normalization for Unconventional Writing of Under-Resourced Languages in Bilingual Communities25 May 2023
- 2022Language Varieties of Italy: Technology Challenges and Opportunities20 Sep 2022