# VSEC: Transformer-based Model for Vietnamese Spelling Correction

arXiv: https://arxiv.org/abs/2111.00640 · HTML: https://mlsanity.com/paper/2111.00640/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 25 references that are in the map.
- First posted on arXiv 1 Nov 2021.

## Builds on (most cited first, 5 of 5)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Spelling Correction as a Foreign Language](https://mlsanity.com/paper/1705.07371/) (2017)
- [Context-Sensitive Malicious Spelling Error Correction](https://mlsanity.com/paper/1901.07688/) (2019)

## Built on by (most cited first, 6 of 6)

- [Crossing Linguistic Horizons: Finetuning and Comprehensive Evaluation of Vietnamese Large Language Models](https://mlsanity.com/paper/2403.02715/) (2024)
- [Reference-Based Post-OCR Processing with LLM for Precise Diacritic Text in Historical Document Recognition](https://mlsanity.com/paper/2410.13305/) (2024)
- [ViSoLex: An Open-Source Repository for Vietnamese Social Media Lexical Normalization](https://mlsanity.com/paper/2501.07020/) (2025)
- [Automatic Textual Normalization for Hate Speech Detection](https://mlsanity.com/paper/2311.06851/) (2023)
- [ViLexNorm: A Lexical Normalization Corpus for Vietnamese Social Media Text](https://mlsanity.com/paper/2401.16403/) (2024)
- [Understanding Tieq Viet with Deep Learning Models](https://mlsanity.com/paper/2207.00975/) (2022)

## Abstract (arXiv)

Spelling error correction is one of topics which have a long history in natural language processing. Although previous studies have achieved remarkable results, challenges still exist. In the Vietnamese language, a state-of-the-art method for the task infers a syllable's context from its adjacent syllables. The method's accuracy can be unsatisfactory, however, because the model may lose the context if two (or more) spelling mistakes stand near each other. In this paper, we propose a novel method to correct Vietnamese spelling errors. We tackle the problems of mistyped errors and misspelled errors by using a deep learning model. The embedding layer, in particular, is powered by the byte pair encoding technique. The sequence to sequence model based on the Transformer architecture makes our approach different from the previous works on the same problem. In the experiment, we train the model with a large synthetic dataset, which is randomly introduced spelling errors. We test the performance of the proposed method using a realistic dataset. This dataset contains 11,202 human-made misspellings in 9,341 different Vietnamese sentences. The experimental results show that our method achieves encouraging performance with 86.8% errors detected and 81.5% errors corrected, which improves the state-of-the-art approach 5.6% and 2.2%, respectively.