RoBERTurk: Adjusting RoBERTa for Turkish
Not on the map. Why?
Abstract (arXiv)
We pretrain RoBERTa on a Turkish corpora using BPE tokenizer. Our model outperforms BERTurk family models on the BOUN dataset for the POS task while resulting in underperformance on the IMST dataset for the same task and achieving competitive scores on the Turkish split of the XTREME dataset for the NER task - all while being pretrained on smaller data than its competitors. We release our pretrained model and tokenizer.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 9 of its 17 references that are in the map. First posted on arXiv 7 Jan 2024.
Builds on
9 of 17 references.
References · 9
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
- 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
- 2020A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languagescited by 131
- 2022Enhancements to the BOUN Treebank Reflecting the Agglutinative Nature of Turkishcited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.