# Monolingual versus Multilingual BERTology for Vietnamese Extractive Multi-Document Summarization

arXiv: https://arxiv.org/abs/2108.13741 · HTML: https://mlsanity.com/paper/2108.13741/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 37 references that are in the map.
- First posted on arXiv 31 Aug 2021.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [PhoBERT: Pre-trained language models for Vietnamese](https://mlsanity.com/paper/2003.00744/) (2020)
- [Discourse-Aware Neural Extractive Text Summarization](https://mlsanity.com/paper/1910.14142/) (2019)
- [Text Summarization Techniques: A Brief Survey](https://mlsanity.com/paper/1707.02268/) (2017)
- [Constructive and Toxic Speech Detection for Open-domain Social Media Comments in Vietnamese](https://mlsanity.com/paper/2103.10069/) (2021)
- [Multi-document abstractive summarization using ILP based multi-sentence compression](https://mlsanity.com/paper/1609.07034/) (2016)
- [An Experimental Study of Deep Neural Network Models for Vietnamese Multiple-Choice Reading Comprehension](https://mlsanity.com/paper/2008.08810/) (2020)

## Built on by (most cited first, 6 of 6)

- [ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generation](https://mlsanity.com/paper/2205.06457/) (2022)
- [Mono vs Multilingual BERT for Hate Speech Detection and Text Classification: A Case Study in Marathi](https://mlsanity.com/paper/2204.08669/) (2022)
- [SMTCE: A Social Media Text Classification Evaluation Benchmark and BERTology Models for Vietnamese](https://mlsanity.com/paper/2209.10482/) (2022)
- [ViMMRC 2.0 -- Enhancing Machine Reading Comprehension on Vietnamese Literature Text](https://mlsanity.com/paper/2303.18162/) (2023)
- [VieSum: How Robust Are Transformer-based Models on Vietnamese Summarization?](https://mlsanity.com/paper/2110.04257/) (2021)
- [BERT-VBD: Vietnamese Multi-Document Summarization Framework](https://mlsanity.com/paper/2409.12134/) (2024)

## Abstract (arXiv)

Recent researches have demonstrated that BERT shows potential in a wide range of natural language processing tasks. It is adopted as an encoder for many state-of-the-art automatic summarizing systems, which achieve excellent performance. However, so far, there is not much work done for Vietnamese. In this paper, we showcase how BERT can be implemented for extractive text summarization in Vietnamese on multi-document. We introduce a novel comparison between different multilingual and monolingual BERT models. The experiment results indicate that monolingual models produce promising results compared to other multilingual models and previous text summarizing models for Vietnamese.