# A Comparative Study on Transfer Learning and Distance Metrics in Semantic Clustering over the COVID-19 Tweets

arXiv: https://arxiv.org/abs/2111.08658 · HTML: https://mlsanity.com/paper/2111.08658/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 43 references that are in the map.
- First posted on arXiv 16 Nov 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [CORD-19: The COVID-19 Open Research Dataset](https://mlsanity.com/paper/2004.10706/) (2020)
- [Deep Sentiment Classification and Topic Discovery on Novel Coronavirus or COVID-19 Online Discussions: NLP Using LSTM Recurrent Neural Network Approach](https://mlsanity.com/paper/2004.11695/) (2020)
- [Rapidly Bootstrapping a Question Answering Dataset for COVID-19](https://mlsanity.com/paper/2004.11339/) (2020)
- [Detecting Topic and Sentiment Dynamics Due to COVID-19 Pandemic Using Social Media](https://mlsanity.com/paper/2007.02304/) (2020)
- [COVID-19 Public Sentiment Insights and Machine Learning for Tweets Classification](https://mlsanity.com/paper/2005.10898/) (2020)
- [Cross-language sentiment analysis of European Twitter messages duringthe COVID-19 pandemic](https://mlsanity.com/paper/2008.12172/) (2020)
- [Advance Warning Methodologies for COVID-19 using Chest X-Ray Images](https://mlsanity.com/paper/2006.05332/) (2020)
- [TweetsCOV19 -- A Knowledge Base of Semantically Annotated Tweets about the COVID-19 Pandemic](https://mlsanity.com/paper/2006.14492/) (2020)
- [CORD19STS: COVID-19 Semantic Textual Similarity Dataset](https://mlsanity.com/paper/2007.02461/) (2020)
- [COVID-19 Twitter Dataset with Latent Topics, Sentiments and Emotions Attributes](https://mlsanity.com/paper/2007.06954/) (2020)
- [TopicBERT: A Transformer transfer learning based memory-graph approach for multimodal streaming social media topic detection](https://mlsanity.com/paper/2008.06877/) (2020)
- [Understanding the perception of COVID-19 policies by mining a multilanguage Twitter dataset](https://mlsanity.com/paper/2003.10359/) (2020)
- [Large-scale, Language-agnostic Discourse Classification of Tweets During COVID-19](https://mlsanity.com/paper/2008.00461/) (2020)

## Built on by (most cited first, 1 of 1)

- [Interpretable Company Similarity with Sparse Autoencoders](https://mlsanity.com/paper/2412.02605/) (2024)

## Abstract (arXiv)

This paper is a comparison study in the context of Topic Detection on COVID-19 data. There are various approaches for Topic Detection, among which the Clustering approach is selected in this paper. Clustering requires distance and calculating distance needs embedding. The aim of this research is to simultaneously study the three factors of embedding methods, distance metrics and clustering methods and their interaction. A dataset including one-month tweets collected with COVID-19-related hashtags is used for this study. Five methods, from earlier to new methods, are selected among the embedding methods: Word2Vec, fastText, GloVe, BERT and T5. Five clustering methods are investigated in this paper that are: k-means, DBSCAN, OPTICS, spectral and Jarvis-Patrick. Euclidian distance and Cosine distance as the most important distance metrics in this field are also examined. First, more than 7,500 tests are performed to tune the parameters. Then, all the different combinations of embedding methods with distance metrics and clustering methods are investigated by silhouette metric. The number of these combinations is 50 cases. First, the results of these 50 tests are examined. Then, the rank of each method is taken into account in all the tests of that method. Finally, the major variables of the research (embedding methods, distance metrics and clustering methods) are studied separately. Averaging is performed over the control variables to neutralize their effect. The experimental results show that T5 strongly outperforms other embedding methods in terms of silhouette metric. In terms of distance metrics, cosine distance is weakly better. DBSCAN is also superior to other methods in terms of clustering methods.