# COVID-19 Tweets Analysis through Transformer Language Models

arXiv: https://arxiv.org/abs/2103.00199 · HTML: https://mlsanity.com/paper/2103.00199/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 16 references that are in the map.
- First posted on arXiv 27 Feb 2021.

## Builds on (most cited first, 7 of 7)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Causal Modeling of Twitter Activity During COVID-19](https://mlsanity.com/paper/2005.07952/) (2020)
- [Dynamic topic modeling of the COVID-19 Twitter narrative among U.S. governors and cabinet executives](https://mlsanity.com/paper/2004.11692/) (2020)
- [Early Outbreak Detection for Proactive Crisis Management Using Twitter Data: COVID-19 a Case Study in the US](https://mlsanity.com/paper/2005.00475/) (2020)
- [A thematic analysis of highly retweeted early COVID -19 tweets: Consensus, information, dissent, and lockdown life](https://mlsanity.com/paper/2004.02793/) (2020)

## Built on by (most cited first, 1 of 1)

- [Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora](https://mlsanity.com/paper/2110.08534/) (2021)

## Abstract (arXiv)

Understanding the public sentiment and perception in a healthcare crisis is essential for developing appropriate crisis management techniques. While some studies have used Twitter data for predictive modelling during COVID-19, fine-grained sentiment analysis of the opinion of people on social media during this pandemic has not yet been done. In this study, we perform an in-depth, fine-grained sentiment analysis of tweets in COVID-19. For this purpose, we perform supervised training of four transformer language models on the downstream task of multi-label classification of tweets into seven tone classes: [confident, anger, fear, joy, sadness, analytical, tentative]. We achieve a LRAP (Label Ranking Average Precision) score of 0.9267 through RoBERTa. This trained transformer model is able to correctly predict, with high accuracy, the tone of a tweet. We then leverage this model for predicting tones for 200,000 tweets on COVID-19. We then perform a country-wise analysis of the tone of tweets, and extract useful indicators of the psychological condition about the people in this pandemic.