# ViLexNorm: A Lexical Normalization Corpus for Vietnamese Social Media Text

arXiv: https://arxiv.org/abs/2401.16403 · HTML: https://mlsanity.com/paper/2401.16403/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 44 references that are in the map.
- First posted on arXiv 29 Jan 2024.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [PhoBERT: Pre-trained language models for Vietnamese](https://mlsanity.com/paper/2003.00744/) (2020)
- [VnCoreNLP: A Vietnamese Natural Language Processing Toolkit](https://mlsanity.com/paper/1801.01331/) (2018)
- [BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese](https://mlsanity.com/paper/2109.09701/) (2021)
- [A Large-scale Dataset for Hate Speech Detection on Vietnamese Social Media Texts](https://mlsanity.com/paper/2103.11528/) (2021)
- [Emotion Recognition for Vietnamese Social Media Text](https://mlsanity.com/paper/1911.09339/) (2019)
- [DaN+: Danish Nested Named Entities and Lexical Normalization](https://mlsanity.com/paper/2105.11301/) (2021)
- [Universal Dependency Parsing for Hindi-English Code-switching](https://mlsanity.com/paper/1804.05868/) (2018)
- [VSEC: Transformer-based Model for Vietnamese Spelling Correction](https://mlsanity.com/paper/2111.00640/) (2021)
- [hinglishNorm -- A Corpus of Hindi-English Code Mixed Sentences for Text Normalization](https://mlsanity.com/paper/2010.08974/) (2020)
- [User-Generated Text Corpus for Evaluating Japanese Morphological Analysis and Lexical Normalization](https://mlsanity.com/paper/2104.03523/) (2021)

## Built on by (most cited first, 1 of 1)

- [A Weakly Supervised Data Labeling Framework for Machine Lexical Normalization in Vietnamese Social Media](https://mlsanity.com/paper/2409.20467/) (2024)

## Abstract (arXiv)

Lexical normalization, a fundamental task in Natural Language Processing (NLP), involves the transformation of words into their canonical forms. This process has been proven to benefit various downstream NLP tasks greatly. In this work, we introduce Vietnamese Lexical Normalization (ViLexNorm), the first-ever corpus developed for the Vietnamese lexical normalization task. The corpus comprises over 10,000 pairs of sentences meticulously annotated by human annotators, sourced from public comments on Vietnam's most popular social media platforms. Various methods were used to evaluate our corpus, and the best-performing system achieved a result of 57.74% using the Error Reduction Rate (ERR) metric (van der Goot, 2019a) with the Leave-As-Is (LAI) baseline. For extrinsic evaluation, employing the model trained on ViLexNorm demonstrates the positive impact of the Vietnamese lexical normalization task on other NLP tasks. Our corpus is publicly available exclusively for research purposes.