mlsanityData to 7 Oct 2026

COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter

arXiv 2005.07503 ↗First posted 15 May 2020cs.CL cs.LG cs.SIcited by 96

Not on the map. Why?

Abstract (arXiv)

In this work, we release COVID-Twitter-BERT (CT-BERT), a transformer-based model, pretrained on a large corpus of Twitter messages on the topic of COVID-19. Our model shows a 10-30% marginal improvement compared to its base model, BERT-Large, on five different classification datasets. The largest improvements are on the target domain. Pretrained transformer models, such as CT-BERT, are trained on a specific target domain and can be used for a wide variety of natural language processing tasks, including classification, question-answering and chatbots. CT-BERT is optimised to be used on COVID-19 content, in particular social media posts from Twitter.

Key facts. Cited by 96 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 23; 2023: 21; 2024: 17; 2025: 9; 2026: 1.

96 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  2. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  3. 2021Lifelong Pretraining: Continually Adapting Language Models to Emerging Corporacited by 114
  4. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  5. 2021Pre-trained Language Models in Biomedical Domain: A Systematic Surveycited by 87
  6. 2023Fighting Fire with Fire: The Dual Role of LLMs in Crafting and Detecting Elusive Disinformationcited by 51
  7. 2021Misinfo Reaction Frames: Reasoning about Readers' Reactions to News Headlinescited by 32
  8. 2021AMMU : A Survey of Transformer-based Biomedical Pretrained Language Modelscited by 32
  9. 2021The Surprising Performance of Simple Baselines for Misinformation Detectioncited by 27
  10. 2022HealthPrompt: A Zero-shot Learning Paradigm for Clinical Natural Language Processingcited by 26
  11. 2022Infusing Knowledge from Wikipedia to Enhance Stance Detectioncited by 20
  12. 2023Don't Waste a Single Annotation: Improving Single-Label Classifiers Through Soft Labelscited by 17
38 more
  1. 2021Evaluating Deep Learning Approaches for Covid19 Fake News Detectioncited by 16
  2. 2022CAVES: A Dataset to facilitate Explainable Classification and Summarization of Concerns towards COVID Vaccinescited by 16
  3. 2021Transformer-based Language Model Fine-tuning Methods for COVID-19 Fake News Detectioncited by 15
  4. 2022Contrastive Domain Adaptation for Early Misinformation Detection: A Case Study on COVID-19cited by 15
  5. 2021Knowledge Graph informed Fake News Classification via Heterogeneous Representation Ensemblescited by 14
  6. 2022Human-in-the-loop Evaluation for Early Misinformation Detection: A Case Study of COVID-19 Treatmentscited by 14
  7. 2023VaxxHesitancy: A Dataset for Studying Hesitancy towards COVID-19 Vaccination on Twittercited by 14
  8. 2023Examining Bias in Opinion Summarisation Through the Perspective of Opinion Diversitycited by 14
  9. 2021A Comparative Study of Transformer-Based Language Models on Extractive Question Answeringcited by 13
  10. 2022METS-CoV: A Dataset of Medical Entity and Targeted Sentiment on COVID-19 Related Tweetscited by 12
  11. 2024Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcarecited by 12
  12. 2021Testing the Generalization of Neural Language Models for COVID-19 Misinformation Detectioncited by 10
  13. 2023QUERT: Continual Pre-training of Language Model for Query Understanding in Travel Domain Searchcited by 10
  14. 2021Categorising Fine-to-Coarse Grained Misinformation: An Empirical Study of COVID-19 Infodemiccited by 9
  15. 2023Shared and Private Information Learning in Multimodal Sentiment Analysis with Deep Modal Alignment and Self-supervised Multi-Task Learningcited by 9
  16. 2025ADMIT: Few-shot Knowledge Poisoning Attacks on RAG-based Fact Checkingcited by 9
  17. 2023Leveraging Large Language Models and Weak Supervision for Social Media data annotation: an evaluation using COVID-19 self-reported vaccination tweetscited by 8
  18. 2024VLUE: A New Benchmark and Multi-task Knowledge Transfer Learning for Vietnamese Natural Language Understandingcited by 8
  19. 2021Automatic Detection of COVID-19 Vaccine Misinformation with Graph Link Predictioncited by 7
  20. 2021Deep Transfer Learning & Beyond: Transformer Language Models in Information Systems Researchcited by 7
  21. 2022VaccineLies: A Natural Language Resource for Learning to Recognize Misinformation about the COVID-19 and HPV Vaccinescited by 7
  22. 2022Benchmarking for Public Health Surveillance tasks on Social Media with a Domain-Specific Pretrained Language Modelcited by 7
  23. 2024Media Bias Matters: Understanding the Impact of Politically Biased News on Vaccine Attitudes in Social Mediacited by 7
  24. 2021On the Role of Images for Analyzing Claims in Social Mediacited by 6
  25. 2022Artificial intelligence for topic modelling in Hindu philosophy: mapping themes between the Upanishads and the Bhagavad Gitacited by 6
  26. 2023Exploring Social Media for Early Detection of Depression in COVID-19 Patientscited by 6
  27. 2023Understanding Social Media Cross-Modality Discourse in Linguistic Spacecited by 6
  28. 2023Balanced and Explainable Social Media Analysis for Public Health with Large Language Modelscited by 6
  29. 2024MegaFake: A Theory-Driven Dataset of Fake News Generated by Large Language Modelscited by 5
  30. 2024Stanceformer: Target-Aware Transformer for Stance Detectioncited by 5
  31. 2021Leveraging cross-platform data to improve automated hate speech detectioncited by 4
  32. 2021Emotion analysis and detection during COVID-19cited by 4
  33. 2022From Hesitancy Framings to Vaccine Hesitancy Profiles: A Journey of Stance, Ontological Commitments and Moral Foundationscited by 4
  34. 2022The COVMis-Stance dataset: Stance Detection on Twitter for COVID-19 Misinformationcited by 4
  35. 2024Detection of Conspiracy Theories Beyond Keyword Bias in German-Language Telegram Using Large Language Modelscited by 4
  36. 2021Fighting the COVID-19 Infodemic with a Holistic BERT Ensemblecited by 3
  37. 2021BERT based Transformers lead the way in Extraction of Health Information from Social Mediacited by 3
  38. 2021Classifying vaccine sentiment tweets by modelling domain-specific representation and commonsense knowledge into context-aware attentive GRUcited by 3

50 most cited of 96 All 96 (JSON)