# Improving Sequence Tagging for Vietnamese Text Using Transformer-based Neural Models

arXiv: https://arxiv.org/abs/2006.15994 · HTML: https://mlsanity.com/paper/2006.15994/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jun 2020.

## Built on by (most cited first, 22 of 22)

- [ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generation](https://mlsanity.com/paper/2205.06457/) (2022)
- [BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese](https://mlsanity.com/paper/2109.09701/) (2021)
- [ViDeBERTa: A powerful pre-trained language model for Vietnamese](https://mlsanity.com/paper/2301.10439/) (2023)
- [ViSoBERT: A Pre-Trained Language Model for Vietnamese Social Media Text Processing](https://mlsanity.com/paper/2310.11166/) (2023)
- [SMTCE: A Social Media Text Classification Evaluation Benchmark and BERTology Models for Vietnamese](https://mlsanity.com/paper/2209.10482/) (2022)
- [ViMMRC 2.0 -- Enhancing Machine Reading Comprehension on Vietnamese Literature Text](https://mlsanity.com/paper/2303.18162/) (2023)
- [Investigating Monolingual and Multilingual BERTModels for Vietnamese Aspect Category Detection](https://mlsanity.com/paper/2103.09519/) (2021)
- [Multi-stage Information Retrieval for Vietnamese Legal Texts](https://mlsanity.com/paper/2209.14494/) (2022)
- [ViHateT5: Enhancing Hate Speech Detection in Vietnamese With A Unified Text-to-Text Transformer Model](https://mlsanity.com/paper/2405.14141/) (2024)
- [R2GQA: Retriever-Reader-Generator Question Answering System to Support Students Understanding Legal Regulations in Higher Education](https://mlsanity.com/paper/2409.02840/) (2024)
- [VieSum: How Robust Are Transformer-based Models on Vietnamese Summarization?](https://mlsanity.com/paper/2110.04257/) (2021)
- [Vietnamese Capitalization and Punctuation Recovery Models](https://mlsanity.com/paper/2207.01312/) (2022)
- [ViCGCN: Graph Convolutional Network with Contextualized Language Models for Social Media Mining in Vietnamese](https://mlsanity.com/paper/2309.02902/) (2023)
- [ViLegalNLI: Natural Language Inference for Vietnamese Legal Texts](https://mlsanity.com/paper/2605.00116/) (2026)
- [ViTHSD: Exploiting Hatred by Targets for Hate Speech Detection on Vietnamese Social Media Texts](https://mlsanity.com/paper/2404.19252/) (2024)
- [A study of Vietnamese readability assessing through semantic and statistical features](https://mlsanity.com/paper/2411.04756/) (2024)
- [Fake Advertisements Detection Using Automated Multimodal Learning: A Case Study for Vietnamese Real Estate Data](https://mlsanity.com/paper/2501.10848/) (2025)
- [Investigating Recent Large Language Models for Vietnamese Machine Reading Comprehension](https://mlsanity.com/paper/2503.18062/) (2025)
- [A Vietnamese Dataset for Text Segmentation and Multiple Choices Reading Comprehension](https://mlsanity.com/paper/2506.15978/) (2025)
- [NEU-ESC: A Comprehensive Vietnamese dataset for Educational Sentiment analysis and topic Classification toward multitask learning](https://mlsanity.com/paper/2506.23524/) (2025)
- [ViGoEmotions: A Benchmark Dataset For Fine-grained Emotion Detection on Vietnamese Texts](https://mlsanity.com/paper/2602.08371/) (2026)
- [BamiBERT: A New BERT-based Language Model for Vietnamese](https://mlsanity.com/paper/2607.02259/) (2026)

## Abstract (arXiv)

This paper describes our study on using mutilingual BERT embeddings and some new neural models for improving sequence tagging tasks for the Vietnamese language. We propose new model architectures and evaluate them extensively on two named entity recognition datasets of VLSP 2016 and VLSP 2018, and on two part-of-speech tagging datasets of VLSP 2010 and VLSP 2013. Our proposed models outperform existing methods and achieve new state-of-the-art results. In particular, we have pushed the accuracy of part-of-speech tagging to 95.40% on the VLSP 2010 corpus, to 96.77% on the VLSP 2013 corpus; and the F1 score of named entity recognition to 94.07% on the VLSP 2016 corpus, to 90.31% on the VLSP 2018 corpus. Our code and pre-trained models viBERT and vELECTRA are released as open source to facilitate adoption and further research.