# SMTCE: A Social Media Text Classification Evaluation Benchmark and BERTology Models for Vietnamese

arXiv: https://arxiv.org/abs/2209.10482 · HTML: https://mlsanity.com/paper/2209.10482/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 24 references that are in the map.
- First posted on arXiv 21 Sep 2022.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [CamemBERT: a Tasty French Language Model](https://mlsanity.com/paper/1911.03894/) (2019)
- [Revisiting Pre-Trained Models for Chinese Natural Language Processing](https://mlsanity.com/paper/2004.13922/) (2020)
- [PhoBERT: Pre-trained language models for Vietnamese](https://mlsanity.com/paper/2003.00744/) (2020)
- [Pre-Training Transformers as Energy-Based Cloze Models](https://mlsanity.com/paper/2012.08561/) (2020)
- [A Large-scale Dataset for Hate Speech Detection on Vietnamese Social Media Texts](https://mlsanity.com/paper/2103.11528/) (2021)
- [Emotion Recognition for Vietnamese Social Media Text](https://mlsanity.com/paper/1911.09339/) (2019)
- [Improving Sequence Tagging for Vietnamese Text Using Transformer-based Neural Models](https://mlsanity.com/paper/2006.15994/) (2020)
- [Constructive and Toxic Speech Detection for Open-domain Social Media Comments in Vietnamese](https://mlsanity.com/paper/2103.10069/) (2021)
- [A Simple and Efficient Ensemble Classifier Combining Multiple Neural Network Models on Social Media Datasets in Vietnamese](https://mlsanity.com/paper/2009.13060/) (2020)
- [VLSP 2021 - ViMRC Challenge: Vietnamese Machine Reading Comprehension](https://mlsanity.com/paper/2203.11400/) (2022)
- [Exploiting Vietnamese Social Media Characteristics for Textual Emotion Recognition in Vietnamese](https://mlsanity.com/paper/2009.11005/) (2020)
- [Monolingual versus Multilingual BERTology for Vietnamese Extractive Multi-Document Summarization](https://mlsanity.com/paper/2108.13741/) (2021)
- [Investigating Monolingual and Multilingual BERTModels for Vietnamese Aspect Category Detection](https://mlsanity.com/paper/2103.09519/) (2021)

## Built on by (most cited first, 7 of 7)

- [ViSoBERT: A Pre-Trained Language Model for Vietnamese Social Media Text Processing](https://mlsanity.com/paper/2310.11166/) (2023)
- [VLUE: A New Benchmark and Multi-task Knowledge Transfer Learning for Vietnamese Natural Language Understanding](https://mlsanity.com/paper/2403.15882/) (2024)
- [ViMMRC 2.0 -- Enhancing Machine Reading Comprehension on Vietnamese Literature Text](https://mlsanity.com/paper/2303.18162/) (2023)
- [ViHateT5: Enhancing Hate Speech Detection in Vietnamese With A Unified Text-to-Text Transformer Model](https://mlsanity.com/paper/2405.14141/) (2024)
- [ViCGCN: Graph Convolutional Network with Contextualized Language Models for Social Media Mining in Vietnamese](https://mlsanity.com/paper/2309.02902/) (2023)
- [Metadata Integration for Spam Reviews Detection on Vietnamese E-commerce Websites](https://mlsanity.com/paper/2405.13292/) (2024)
- [ViTHSD: Exploiting Hatred by Targets for Hate Speech Detection on Vietnamese Social Media Texts](https://mlsanity.com/paper/2404.19252/) (2024)

## Abstract (arXiv)

Text classification is a typical natural language processing or computational linguistics task with various interesting applications. As the number of users on social media platforms increases, data acceleration promotes emerging studies on Social Media Text Classification (SMTC) or social media text mining on these valuable resources. In contrast to English, Vietnamese, one of the low-resource languages, is still not concentrated on and exploited thoroughly. Inspired by the success of the GLUE, we introduce the Social Media Text Classification Evaluation (SMTCE) benchmark, as a collection of datasets and models across a diverse set of SMTC tasks. With the proposed benchmark, we implement and analyze the effectiveness of a variety of multilingual BERT-based models (mBERT, XLM-R, and DistilmBERT) and monolingual BERT-based models (PhoBERT, viBERT, vELECTRA, and viBERT4news) for tasks in the SMTCE benchmark. Monolingual models outperform multilingual models and achieve state-of-the-art results on all text classification tasks. It provides an objective assessment of multilingual and monolingual BERT-based models on the benchmark, which will benefit future studies about BERTology in the Vietnamese language.