mlsanityData to 7 Oct 2026

BERTweet: A pre-trained language model for English Tweets

arXiv 2005.10200 ↗First posted 20 May 2020cs.CL cs.LGcited by 346

Not on the map. Why?

Abstract (arXiv)

We present BERTweet, the first public large-scale pre-trained language model for English Tweets. Our BERTweet, having the same architecture as BERT-base (Devlin et al., 2019), is trained using the RoBERTa pre-training procedure (Liu et al., 2019). Experiments show that BERTweet outperforms strong baselines RoBERTa-base and XLM-R-base (Conneau et al., 2020), producing better performance results than the previous state-of-the-art models on three Tweet NLP tasks: Part-of-speech tagging, Named-entity recognition and text classification. We release BERTweet under the MIT License to facilitate future research and applications on Tweet data. Our BERTweet is available at https://github.com/VinAIResearch/BERTweet

Key facts. Cited by 346 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 57; 2022: 79; 2023: 96; 2024: 63; 2025: 34; 2026: 17.

346 papers.

Sort:

By area:

  1. 2023BloombergGPT: A Large Language Model for Financecited by 923
  2. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  3. 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 181
  4. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  5. 2022TimeLMs: Diachronic Language Models from Twittercited by 129
  6. 2022Jury Learning: Integrating Dissenting Voices into Machine Learning Modelscited by 115
  7. 2021XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyondcited by 114
  8. 2021Lifelong Pretraining: Continually Adapting Language Models to Emerging Corporacited by 114
  9. 2021Pre-trained Language Models in Biomedical Domain: A Systematic Surveycited by 87
  10. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 86
  11. 2023A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4cited by 81
  12. 2021Improving Named Entity Recognition by External Context Retrieving and Cooperative Learningcited by 76
38 more
  1. 2022TweetNLP: Cutting-Edge Natural Language Processing for Social Mediacited by 71
  2. 2022Handling Bias in Toxic Speech Detection: A Surveycited by 60
  3. 2023A Bibliometric Review of Large Language Models Research from 2017 to 2023cited by 56
  4. 2023Bias Against 93 Stigmatized Groups in Masked Language Models and Downstream Sentiment Classification Taskscited by 51
  5. 2021Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hatecited by 50
  6. 2023SemEval-2023 Task 10: Explainable Detection of Online Sexismcited by 48
  7. 2022TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twittercited by 44
  8. 2023Exploring Cross-Cultural Differences in English Hate Speech Annotations: From Dataset Construction to Analysiscited by 44
  9. 2024eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Datacited by 44
  10. 2023WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Modelscited by 42
  11. 2021Exploiting BERT For Multimodal Target Sentiment Classification Through Input Space Translationcited by 41
  12. 2022SkillSpan: Hard and Soft Skill Extraction from English Job Postingscited by 41
  13. 2022Large-Scale Hate Speech Detection with Cross-Domain Transfercited by 39
  14. 2021Your fairness may vary: Pretrained language model fairness in toxic text classificationcited by 38
  15. 2023SentimentGPT: Exploiting GPT for Advanced Sentiment Analysis and its Departure from Current Machine Learningcited by 38
  16. 2022CommunityLM: Probing Partisan Worldviews from Language Modelscited by 36
  17. 2021Robust Transfer Learning with Pretrained Language Models through Adapterscited by 35
  18. 2024Claim Detection for Automated Fact-checking: A Survey on Monolingual, Multilingual and Cross-Lingual Researchcited by 35
  19. 2021AMMU : A Survey of Transformer-based Biomedical Pretrained Language Modelscited by 32
  20. 2021FBERT: A Neural Transformer for Identifying Offensive Contentcited by 30
  21. 2021Accelerating BERT Inference for Sequence Labeling via Early-Exitcited by 29
  22. 2023Robust Hate Speech Detection in Social Media: A Cross-Dataset Empirical Evaluationcited by 28
  23. 2021The Surprising Performance of Simple Baselines for Misinformation Detectioncited by 27
  24. 2021Dynamic Language Models for Continuously Evolving Contentcited by 27
  25. 2022Named Entity Recognition in Twitter: A Dataset and Analysis on Short-Term Temporal Shiftscited by 27
  26. 2021A systematic review of Hate Speech automatic detection using Natural Language Processingcited by 26
  27. 2021IndoBERTweet: A Pretrained Language Model for Indonesian Twitter with Effective Domain-Specific Vocabulary Initializationcited by 25
  28. 2021RoBERTuito: a pre-trained language model for social media text in Spanishcited by 22
  29. 2022Twitter Topic Classificationcited by 22
  30. 2022On the Domain Adaptation and Generalization of Pretrained Language Models: A Surveycited by 22
  31. 2022Infusing Knowledge from Wikipedia to Enhance Stance Detectioncited by 20
  32. 2022Modeling Motivational Interviewing Strategies On An Online Peer-to-Peer Counseling Platformcited by 20
  33. 2023Stance Detection: A Practical Guide to Classifying Political Beliefs in Textcited by 20
  34. 2025Harnessing Large Language Models for Disaster Management: A Surveycited by 20
  35. 2022TempoWiC: An Evaluation Benchmark for Detecting Meaning Shift in Social Mediacited by 19
  36. 2024Multi-modal Stance Detection: New Datasets and Modelcited by 18
  37. 2022Location reference recognition from texts: A survey and comparisoncited by 17
  38. 2023LMBot: Distilling Graph Knowledge into Language Model for Graph-less Deployment in Twitter Bot Detectioncited by 17

50 most cited of 346 All 346 (JSON)