# Short text classification with machine learning in the social sciences: The case of climate change on Twitter

arXiv: https://arxiv.org/abs/2310.04452 · HTML: https://mlsanity.com/paper/2310.04452/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 97 references that are in the map.
- First posted on arXiv 3 Oct 2023.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Climate change sentiment on Twitter: An unsolicited public opinion poll](https://mlsanity.com/paper/1505.03804/) (2015)
- [Toward Optimal Feature Selection in Naive Bayes for Text Categorization](https://mlsanity.com/paper/1602.02850/) (2016)
- [Multiple decision trees](https://mlsanity.com/paper/1304.2363/) (2013)
- [On the Performance of Differential Evolution for Hyperparameter Tuning](https://mlsanity.com/paper/1904.06960/) (2019)

## Built on by (most cited first, 1 of 1)

- [The Language of Weather: Social Media Reactions to Weather Accounting for Climatic and Linguistic Baselines](https://mlsanity.com/paper/2407.07683/) (2024)

## Abstract (arXiv)

To analyse large numbers of texts, social science researchers are increasingly confronting the challenge of text classification. When manual labeling is not possible and researchers have to find automatized ways to classify texts, computer science provides a useful toolbox of machine-learning methods whose performance remains understudied in the social sciences. In this article, we compare the performance of the most widely used text classifiers by applying them to a typical research scenario in social science research: a relatively small labeled dataset with infrequent occurrence of categories of interest, which is a part of a large unlabeled dataset. As an example case, we look at Twitter communication regarding climate change, a topic of increasing scholarly interest in interdisciplinary social science research. Using a novel dataset including 5,750 tweets from various international organizations regarding the highly ambiguous concept of climate change, we evaluate the performance of methods in automatically classifying tweets based on whether they are about climate change or not. In this context, we highlight two main findings. First, supervised machine-learning methods perform better than state-of-the-art lexicons, in particular as class balance increases. Second, traditional machine-learning methods, such as logistic regression and random forest, perform similarly to sophisticated deep-learning methods, whilst requiring much less training time and computational resources. The results have important implications for the analysis of short texts in social science research.