Data to 7 Oct 2026

PhoBERT: Pre-trained language models for Vietnamese

arXiv 2003.00744 ↗First posted 2 Mar 2020cs.CL cs.AIcited by 148

Not on the map. Why?

Abstract (arXiv)

We present PhoBERT with two versions, PhoBERT-base and PhoBERT-large, the first public large-scale monolingual language models pre-trained for Vietnamese. Experimental results show that PhoBERT consistently outperforms the recent best pre-trained multilingual model XLM-R (Conneau et al., 2020) and improves the state-of-the-art in multiple Vietnamese-specific NLP tasks including Part-of-speech tagging, Dependency parsing, Named-entity recognition and Natural language inference. We release PhoBERT to facilitate future research and downstream applications for Vietnamese NLP. Our PhoBERT models are available at https://github.com/VinAIResearch/PhoBERT

Key facts. Cited by 148 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 29; 2022: 22; 2023: 23; 2024: 31; 2025: 22; 2026: 21.

148 papers.

Sort:

By area:

2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 207
2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
2021Language Models are Few-shot Multilingual Learnerscited by 92
2021What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformerscited by 75
2021Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processingcited by 68
2022L3Cube-MahaCorpus and MahaBERT: Marathi Monolingual Corpus, Marathi BERT Language Models, and Resourcescited by 49
2022NusaCrowd: Open Source Initiative for Indonesian NLP Resourcescited by 38
2022ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generationcited by 37
2021MarIA: Spanish Language Modelscited by 36
2021BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamesecited by 33
2023Typhoon: Thai Large Language Modelscited by 28
2022KinyaBERT: a Morphology-aware Kinyarwanda Language Modelcited by 27

38 more

2023A Family of Pretrained Transformer Language Models for Russiancited by 27
2022MultiSpider: Towards Benchmarking Multilingual Text-to-SQL Semantic Parsingcited by 22
2021Large-Scale Contextualised Language Modelling for Norwegiancited by 21
2021Intent Detection and Slot Filling for Vietnamesecited by 19
2021COVID-19 Named Entity Recognition for Vietnamesecited by 19
2023Sabi'a: Portuguese Large Language Modelscited by 19
2021Constructive and Toxic Speech Detection for Open-domain Social Media Comments in Vietnamesecited by 17
2022Mono vs Multilingual BERT for Hate Speech Detection and Text Classification: A Case Study in Marathicited by 16
2023XPhoneBERT: A Pre-trained Multilingual Model for Phoneme Representations for Text-to-Speechcited by 16
2022BERTifying Sinhala -- A Comprehensive Analysis of Pre-trained Language Models for Sinhala Text Classificationcited by 15
2024Cross-Data Knowledge Graph Construction for LLM-enabled Educational Question-Answering System: A Case Study at HCMUTcited by 14
2021PhoNLP: A joint multi-task learning model for Vietnamese part-of-speech tagging, named entity recognition and dependency parsingcited by 13
2021A Review of Bangla Natural Language Processing Tasks and the Utility of Transformer Modelscited by 13
2023ViHOS: Hate Speech Spans Detection for Vietnamesecited by 13
2023ViDeBERTa: A powerful pre-trained language model for Vietnamesecited by 11
2023ViSoBERT: A Pre-Trained Language Model for Vietnamese Social Media Text Processingcited by 11
2021Sentence Extraction-Based Machine Reading Comprehension for Vietnamesecited by 9
2022A Summary of the ALQAC 2021 Competitioncited by 9
2022Vietnamese Hate and Offensive Detection using PhoBERT-CNN and Social Media Streaming Datacited by 9
2023PhoGPT: Generative Pre-training for Vietnamesecited by 8
2024EthioLLM: Multilingual Large Language Models for Ethiopian Languages with Task Evaluationcited by 8
2024VLUE: A New Benchmark and Multi-task Knowledge Transfer Learning for Vietnamese Natural Language Understandingcited by 8
2025Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asiacited by 8
2021Conversational Machine Reading Comprehension for Vietnamese Healthcare Textscited by 7
2022SMTCE: A Social Media Text Classification Evaluation Benchmark and BERTology Models for Vietnamesecited by 7
2025Toward a Vision-Language Foundation Model for Medical Data: Multimodal Dataset and Benchmarks for Vietnamese PET/CT Report Generationcited by 7
2021Introduction to Neural Transfer Learning with Transformers for Social Science Text Analysiscited by 6
2021Monolingual versus Multilingual BERTology for Vietnamese Extractive Multi-Document Summarizationcited by 6
2024User-Aware Multilingual Abusive Content Detection in Social Mediacited by 6
2021On the ability of monolingual models to learn language-agnostic representationscited by 5
2022SPBERTQA: A Two-Stage Question Answering System Based on Sentence Transformers for Medical Textscited by 5
2023BARTPhoBEiT: Pre-trained Sequence-to-Sequence and Image Transformers Models for Vietnamese Visual Question Answeringcited by 5
2021NLPBK at VLSP-2020 shared task: Compose transformer pretrained models for Reliable Intelligence Identification on Social networkcited by 4
2021Investigating Monolingual and Multilingual BERTModels for Vietnamese Aspect Category Detectioncited by 4
2021Vietnamese Complaint Detection on E-Commerce Websitescited by 4
2022Detecting Spam Reviews on Vietnamese E-commerce Websitescited by 4
2022UIT-ViCoV19QA: A Dataset for COVID-19 Community-based Question Answering on Vietnamese Languagecited by 4
2022Enriching Biomedical Knowledge for Low-resource Language Through Large-Scale Translationcited by 4

50 most cited of 148 All 148 (JSON)