# Utilizing Character and Word Embeddings for Text Normalization with Sequence-to-Sequence Models

arXiv: https://arxiv.org/abs/1809.01534 · HTML: https://mlsanity.com/paper/1809.01534/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Sep 2018.

## Built on by (most cited first, 7 of 7)

- [Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigation](https://mlsanity.com/paper/2305.14734/) (2023)
- [Beyond English: Evaluating LLMs for Arabic Grammatical Error Correction](https://mlsanity.com/paper/2312.08400/) (2023)
- [Enhancing Text Editing for Grammatical Error Correction: Arabic as a Case Study](https://mlsanity.com/paper/2503.00985/) (2025)
- [Exploiting Dialect Identification in Automatic Dialectal Text Normalization](https://mlsanity.com/paper/2407.03020/) (2024)
- [ChatGPT for Arabic Grammatical Error Correction](https://mlsanity.com/paper/2308.04492/) (2023)
- [Correcting Arabic Soft Spelling Mistakes using BiLSTM-based Machine Learning](https://mlsanity.com/paper/2108.01141/) (2021)
- [Pre-Editorial Normalization for Automatically Transcribed Medieval Manuscripts in Old French and Latin](https://mlsanity.com/paper/2602.13905/) (2026)

## Abstract (arXiv)

Text normalization is an important enabling technology for several NLP tasks. Recently, neural-network-based approaches have outperformed well-established models in this task. However, in languages other than English, there has been little exploration in this direction. Both the scarcity of annotated data and the complexity of the language increase the difficulty of the problem. To address these challenges, we use a sequence-to-sequence model with character-based attention, which in addition to its self-learned character embeddings, uses word embeddings pre-trained with an approach that also models subword information. This provides the neural model with access to more linguistic information especially suitable for text normalization, without large parallel corpora. We show that providing the model with word-level features bridges the gap for the neural network approach to achieve a state-of-the-art F1 score on a standard Arabic language correction shared task dataset.