ViLexNorm: A Lexical Normalization Corpus for Vietnamese Social Media Text
Not on the map. Why?
Abstract (arXiv)
Lexical normalization, a fundamental task in Natural Language Processing (NLP), involves the transformation of words into their canonical forms. This process has been proven to benefit various downstream NLP tasks greatly. In this work, we introduce Vietnamese Lexical Normalization (ViLexNorm), the first-ever corpus developed for the Vietnamese lexical normalization task. The corpus comprises over 10,000 pairs of sentences meticulously annotated by human annotators, sourced from public comments on Vietnam's most popular social media platforms. Various methods were used to evaluate our corpus, and the best-performing system achieved a result of 57.74% using the Error Reduction Rate (ERR) metric (van der Goot, 2019a) with the Leave-As-Is (LAI) baseline. For extrinsic evaluation, employing the model trained on ViLexNorm demonstrates the positive impact of the Vietnamese lexical normalization task on other NLP tasks. Our corpus is publicly available exclusively for research purposes.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 16 of its 44 references that are in the map. First posted on arXiv 29 Jan 2024.
Builds on
16 of 44 references.
References · 16
- 2017Attention Is All You Needcited by 59,932
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
- 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
- 2016Bag of Tricks for Efficient Text Classificationcited by 869
- 2020PhoBERT: Pre-trained language models for Vietnamesecited by 148
- 2018VnCoreNLP: A Vietnamese Natural Language Processing Toolkitcited by 51
- 2021BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamesecited by 33
- 2021A Large-scale Dataset for Hate Speech Detection on Vietnamese Social Media Textscited by 25
- 2019Emotion Recognition for Vietnamese Social Media Textcited by 23
- 2021DaN+: Danish Nested Named Entities and Lexical Normalizationcited by 19
4 more
- 2018Universal Dependency Parsing for Hindi-English Code-switchingcited by 18
- 2021VSEC: Transformer-based Model for Vietnamese Spelling Correctioncited by 6
- 2020hinglishNorm -- A Corpus of Hindi-English Code Mixed Sentences for Text Normalizationcited by 3
- 2021User-Generated Text Corpus for Evaluating Japanese Morphological Analysis and Lexical Normalizationcited by 2
Built on by
Citing papers per year: 2024: 1.
1 paper.