# TurkishBERTweet: Fast and Reliable Large Language Model for Social Media Analysis

arXiv: https://arxiv.org/abs/2311.18063 · HTML: https://mlsanity.com/paper/2311.18063/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 48 references that are in the map.
- First posted on arXiv 29 Nov 2023.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity](https://mlsanity.com/paper/2305.13169/) (2023)
- [Towards a Cleaner Document-Oriented Multilingual Crawled Corpus](https://mlsanity.com/paper/2201.06642/) (2022)
- [A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages](https://mlsanity.com/paper/2006.06202/) (2020)
- [ConvBERT: Improving BERT with Span-based Dynamic Convolution](https://mlsanity.com/paper/2008.02496/) (2020)
- [Sentiment of Emojis](https://mlsanity.com/paper/1509.07761/) (2015)
- [Arming the public with artificial intelligence to counter social bots](https://mlsanity.com/paper/1901.00912/) (2019)
- [Just Another Day on Twitter: A Complete 24 Hours of Twitter Data](https://mlsanity.com/paper/2301.11429/) (2023)
- [Developing and Evaluating Tiny to Medium-Sized Turkish BERT Models](https://mlsanity.com/paper/2307.14134/) (2023)
- [Cashtag piggybacking: uncovering spam and bot activity in stock microblogs on Twitter](https://mlsanity.com/paper/1804.04406/) (2018)
- [A Tweet Dataset Annotated for Named Entity Recognition and Stance Detection](https://mlsanity.com/paper/1901.04787/) (2019)
- [#Secim2023: First Public Dataset for Studying Turkish General Election](https://mlsanity.com/paper/2211.13121/) (2022)

## Built on by (most cited first, 4 of 4)

- [Advancing Depression Detection on Social Media Platforms Through Fine-Tuned Large Language Models](https://mlsanity.com/paper/2409.14794/) (2024)
- [MoganBert-TR: A Turkish Encoder Foundation Model Trained from Scratch with a CLM-to-MLM Curriculum](https://mlsanity.com/paper/2608.25768/) (2026)
- [EdgeAIGuard: Agentic LLMs for Minor Protection in Digital Spaces](https://mlsanity.com/paper/2503.00092/) (2025)
- [TabiBERT: A Large-Scale ModernBERT Foundation Model and A Unified Benchmark for Turkish](https://mlsanity.com/paper/2512.23065/) (2025)

## Abstract (arXiv)

Turkish is one of the most popular languages in the world. Wide us of this language on social media platforms such as Twitter, Instagram, or Tiktok and strategic position of the country in the world politics makes it appealing for the social network researchers and industry. To address this need, we introduce TurkishBERTweet, the first large scale pre-trained language model for Turkish social media built using almost 900 million tweets. The model shares the same architecture as base BERT model with smaller input length, making TurkishBERTweet lighter than BERTurk and can have significantly lower inference time. We trained our model using the same approach for RoBERTa model and evaluated on two text classification tasks: Sentiment Classification and Hate Speech Detection. We demonstrate that TurkishBERTweet outperforms the other available alternatives on generalizability and its lower inference time gives significant advantage to process large-scale datasets. We also compared our models with the commercial OpenAI solutions in terms of cost and performance to demonstrate TurkishBERTweet is scalable and cost-effective solution. As part of our research, we released TurkishBERTweet and fine-tuned LoRA adapters for the mentioned tasks under the MIT License to facilitate future research and applications on Turkish social media. Our TurkishBERTweet model is available at: https://github.com/ViralLab/TurkishBERTweet