mlsanityData to 14 Oct 2026

UzBERT: pretraining a BERT model for Uzbek

arXiv 2108.09814 ↗First posted 22 Aug 2021cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Pretrained language models based on the Transformer architecture have achieved state-of-the-art results in various natural language processing tasks such as part-of-speech tagging, named entity recognition, and question answering. However, no such monolingual model for the Uzbek language is publicly available. In this paper, we introduce UzBERT, a pretrained Uzbek language model based on the BERT architecture. Our model greatly outperforms multilingual BERT on masked language model accuracy. We make the model publicly available under the MIT open-source license.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 18 of its 24 references that are in the map. First posted on arXiv 22 Aug 2021.

Builds on

18 of 24 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  6. 2018Deep contextualized word representationscited by 2,290
  7. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  8. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  9. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  10. 2019How multilingual is Multilingual BERT?cited by 645
  11. 2019CamemBERT: a Tasty French Language Modelcited by 368
  12. 2018Learning Word Vectors for 157 Languagescited by 280
6 more
  1. 2019Multilingual is not enough: BERT for Finnishcited by 119
  2. 2013Polyglot: Distributed Word Representations for Multilingual NLPcited by 19
  3. 2020KR-BERT: A Small-Scale Korean-Specific Language Modelcited by 17
  4. 2020Cross-Lingual Word Embeddings for Turkic Languagescited by 12
  5. 2021Uzbek Cyrillic-Latin-Cyrillic Machine Transliterationcited by 4
  6. 2020Development of Word Embeddings for Uzbek Languagecited by 1

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 0; 2024: 1.

5 papers.

  1. 2022SimRelUz: Similarity and Relatedness scores as a Semantic Evaluation dataset for Uzbek languagecited by 10
  2. 2022A machine transliteration tool between Uzbek alphabetscited by 8
  3. 2024Recent Advancements and Challenges of Turkic Central Asian Language Processingcited by 6
  4. 2022Uzbek Sentiment Analysis based on local Restaurant Reviewscited by 3
  5. 2022Creating a morphological and syntactic tagged corpus for the Uzbek languagecited by 1