mlsanityData to 14 Oct 2026

Text normalization for low-resource languages: the case of Ligurian

arXiv 2206.07861 ↗First posted 16 Jun 2022cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Text normalization is a crucial technology for low-resource languages which lack rigid spelling conventions or that have undergone multiple spelling reforms. Low-resource text normalization has so far relied upon hand-crafted rules, which are perceived to be more data efficient than neural methods. In this paper we examine the case of text normalization for Ligurian, an endangered Romance language. We collect 4,394 Ligurian sentences paired with their normalized versions, as well as the first open source monolingual corpus for Ligurian. We show that, in spite of the small amounts of data available, a compact transformer-based model can be trained to achieve very low error rates by the use of backtranslation and appropriate tokenization.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 12 of its 35 references that are in the map. First posted on arXiv 16 Jun 2022.

Builds on

12 of 35 references.

References · 12

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  3. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  4. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  5. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  6. 2020Participatory Research for Low-resourced Machine Translation: A Case Study in African Languagescited by 159
  7. 2020Applying the Transformer to Character-level Transductioncited by 43
  8. 2019A Large-Scale Comparison of Historical Text Normalization Systemscited by 25
  9. 2019Exact Hard Monotonic Attention for Character-Level Transductioncited by 22
  10. 2021Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?cited by 11
  11. 2018An Evaluation of Neural Machine Translation Models on Historical Spelling Normalizationcited by 5
  12. 2021Text Normalization for Low-Resource Languages of Africacited by 2

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 0; 2025: 2.

4 papers.