# Sentiment Analysis of Users' Reviews on COVID-19 Contact Tracing Apps with a Benchmark Dataset

arXiv: https://arxiv.org/abs/2103.01196 · HTML: https://mlsanity.com/paper/2103.01196/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 40 references that are in the map.
- First posted on arXiv 1 Mar 2021.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Collaborative Federated Learning For Healthcare: Multi-Modal COVID-19 Diagnosis at the Edge](https://mlsanity.com/paper/2101.07511/) (2021)
- [Natural Disasters Detection in Social Media and Satellite imagery: a survey](https://mlsanity.com/paper/1901.04277/) (2019)
- [Fake News Detection in Social Media using Graph Neural Networks and NLP Techniques: A COVID-19 Use-case](https://mlsanity.com/paper/2012.07517/) (2020)
- [Model Generalization on COVID-19 Fake News Detection](https://mlsanity.com/paper/2101.03841/) (2021)
- [Contact Tracing Mobile Apps for COVID-19: Privacy Considerations and Related Trade-offs](https://mlsanity.com/paper/2003.11511/) (2020)
- [A Survey of COVID-19 Contact Tracing Apps](https://mlsanity.com/paper/2006.10306/) (2020)
- [COVID-19 Contact-tracing Apps: a Survey on the Global Deployment and Challenges](https://mlsanity.com/paper/2005.03599/) (2020)
- [Early Outbreak Detection for Proactive Crisis Management Using Twitter Data: COVID-19 a Case Study in the US](https://mlsanity.com/paper/2005.00475/) (2020)
- [WeTrace -- A Privacy-preserving Mobile COVID-19 Tracing Approach and Application](https://mlsanity.com/paper/2004.08812/) (2020)
- [Mining user reviews of COVID contact-tracing apps: An exploratory analysis of nine European apps](https://mlsanity.com/paper/2012.13589/) (2020)
- [COVID-19 Digital Contact Tracing Applications and Techniques: A Review Post Initial Deployments](https://mlsanity.com/paper/2103.01766/) (2021)

## Built on by (most cited first, 2 of 2)

- [Artificial Intelligence Based Prognostic Maintenance of Renewable Energy Systems: A Review of Techniques, Challenges, and Future Research Directions](https://mlsanity.com/paper/2104.12561/) (2021)
- [Social Media Informatics for Sustainable Cities and Societies: An Overview of the Applications, associated Challenges, and Potential Solutions](https://mlsanity.com/paper/2412.03600/) (2024)

## Abstract (arXiv)

Contact tracing has been globally adopted in the fight to control the infection rate of COVID-19. Thanks to digital technologies, such as smartphones and wearable devices, contacts of COVID-19 patients can be easily traced and informed about their potential exposure to the virus. To this aim, several interesting mobile applications have been developed. However, there are ever-growing concerns over the working mechanism and performance of these applications. The literature already provides some interesting exploratory studies on the community's response to the applications by analyzing information from different sources, such as news and users' reviews of the applications. However, to the best of our knowledge, there is no existing solution that automatically analyzes users' reviews and extracts the evoked sentiments. In this work, we propose a pipeline starting from manual annotation via a crowd-sourcing study and concluding on the development and training of AI models for automatic sentiment analysis of users' reviews. In total, we employ eight different methods achieving up to an average F1-Scores 94.8% indicating the feasibility of automatic sentiment analysis of users' reviews on the COVID-19 contact tracing applications. We also highlight the key advantages, drawbacks, and users' concerns over the applications. Moreover, we also collect and annotate a large-scale dataset composed of 34,534 reviews manually annotated from the contract tracing applications of 46 distinct countries. The presented analysis and the dataset are expected to provide a baseline/benchmark for future research in the domain.