mlsanityData to 7 Oct 2026

BamiBERT: A New BERT-based Language Model for Vietnamese

arXiv 2607.02259 ↗First posted 2 Jul 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce BamiBERT, a new BERT-based pre-trained language model for Vietnamese that addresses key limitations of PhoBERT -- the current de facto Vietnamese text encoder. Trained from scratch on a 129GB corpus of general-domain Vietnamese text for 20 epochs, BamiBERT supports an extended context length of up to 2048 tokens and operates directly on raw input, eliminating the need for external word segmentation. Across 8 Vietnamese benchmarks, it achieves the best score on 11 of 15 metrics and the second-best on 3 others, setting a new state of the art among "base"-sized Vietnamese encoders and demonstrating strong cross-domain generalization. We release BamiBERT at: https://huggingface.co/Qualcomm-AI-Research/BamiBERT

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 24 references that are in the map. First posted on arXiv 2 Jul 2026.

Builds on

13 of 24 references.

References · 13

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
  6. 2020PhoBERT: Pre-trained language models for Vietnamesecited by 148
  7. 2018VnCoreNLP: A Vietnamese Natural Language Processing Toolkitcited by 51
  8. 2020Improving Sequence Tagging for Vietnamese Text Using Transformer-based Neural Modelscited by 22
  9. 2017A Fast and Accurate Vietnamese Word Segmentercited by 21
  10. 2021COVID-19 Named Entity Recognition for Vietnamesecited by 19
  11. 2021SA2SL: From Aspect-Based Sentiment Analysis to Social Listening System for Business Intelligencecited by 12
  12. 2023ViDeBERTa: A powerful pre-trained language model for Vietnamesecited by 11
1 more
  1. 2023ViSoBERT: A Pre-Trained Language Model for Vietnamese Social Media Text Processingcited by 11

Built on by

Not cited yet.