# Deep learning for COVID-19 topic modelling via Twitter: Alpha, Delta and Omicron

arXiv: https://arxiv.org/abs/2303.00135 · HTML: https://mlsanity.com/paper/2303.00135/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 125 references that are in the map.
- First posted on arXiv 28 Feb 2023.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Top2Vec: Distributed Representations of Topics](https://mlsanity.com/paper/2008.09470/) (2020)
- [Accelerated Hierarchical Density Clustering](https://mlsanity.com/paper/1705.07321/) (2017)
- [Latent Dirichlet Allocation (LDA) and Topic modeling: models, applications, a survey](https://mlsanity.com/paper/1711.04305/) (2017)
- [Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence](https://mlsanity.com/paper/2107.02173/) (2021)
- [Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!](https://mlsanity.com/paper/2004.14914/) (2020)
- [The COVID-19 Social Media Infodemic](https://mlsanity.com/paper/2003.05004/) (2020)
- [Topic Modeling with Contextualized Word Representation Clusters](https://mlsanity.com/paper/2010.12626/) (2020)
- [COVID-19 sentiment analysis via deep learning during the rise of novel cases](https://mlsanity.com/paper/2104.10662/) (2021)
- [Semantic and sentiment analysis of selected Bhagavad Gita translations using BERT-based language framework](https://mlsanity.com/paper/2201.03115/) (2022)
- [Artificial intelligence for topic modelling in Hindu philosophy: mapping themes between the Upanishads and the Bhagavad Gita](https://mlsanity.com/paper/2205.11020/) (2022)
- [Unsupervised machine learning framework for discriminating major variants of concern during COVID-19](https://mlsanity.com/paper/2208.01439/) (2022)

## Built on by (most cited first, 4 of 4)

- [An analysis of vaccine-related sentiments from development to deployment of COVID-19 vaccines](https://mlsanity.com/paper/2306.13797/) (2023)
- [A longitudinal sentiment analysis of Sinophobia during COVID-19 using large language models](https://mlsanity.com/paper/2408.16942/) (2024)
- [HP-BERT: A framework for longitudinal study of Hinduphobia on social media via language models](https://mlsanity.com/paper/2501.05482/) (2025)
- [An evaluation of LLMs for political bias in Western media: Israel-Hamas and Ukraine-Russia wars](https://mlsanity.com/paper/2601.06132/) (2026)

## Abstract (arXiv)

Topic modelling with innovative deep learning methods has gained interest for a wide range of applications that includes COVID-19. Topic modelling can provide, psychological, social and cultural insights for understanding human behaviour in extreme events such as the COVID-19 pandemic. In this paper, we use prominent deep learning-based language models for COVID-19 topic modelling taking into account data from emergence (Alpha) to the Omicron variant. We apply topic modeling to review the public behaviour across the first, second and third waves based on Twitter dataset from India. Our results show that the topics extracted for the subsequent waves had certain overlapping themes such as covers governance, vaccination, and pandemic management while novel issues aroused in political, social and economic situation during COVID-19 pandemic. We also found a strong correlation of the major topics qualitatively to news media prevalent at the respective time period. Hence, our framework has the potential to capture major issues arising during different phases of the COVID-19 pandemic which can be extended to other countries and regions.