# Tweet Sentiment Quantification: An Experimental Re-Evaluation

arXiv: https://arxiv.org/abs/2011.08091 · HTML: https://mlsanity.com/paper/2011.08091/

## Key facts

- Cited by 7 papers in the map (data through 13 Oct 2026).
- First posted on arXiv 4 Nov 2020.

## Built on by (most cited first, 7 of 7)

- [QuaPy: A Python-Based Framework for Quantification](https://mlsanity.com/paper/2106.11057/) (2021)
- [Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach](https://mlsanity.com/paper/2109.08549/) (2021)
- [Kernel Density Estimation for Multiclass Quantification](https://mlsanity.com/paper/2401.00490/) (2023)
- [Binary Quantification and Dataset Shift: An Experimental Investigation](https://mlsanity.com/paper/2310.04565/) (2023)
- [Quantification using Permutation-Invariant Networks based on Histograms](https://mlsanity.com/paper/2403.15123/) (2024)
- [On the Interconnections of Calibration, Quantification, and Classifier Accuracy Prediction under Dataset Shift](https://mlsanity.com/paper/2505.11380/) (2025)
- [Multi-Label Quantification](https://mlsanity.com/paper/2211.08063/) (2022)

## Abstract (arXiv)

Sentiment quantification is the task of training, by means of supervised learning, estimators of the relative frequency (also called ``prevalence'') of sentiment-related classes (such as textsfPositive, textsfNeutral, textsfNegative) in a sample of unlabelled texts. This task is especially important when these texts are tweets, since the final goal of most sentiment classification efforts carried out on Twitter data is actually quantification (and not the classification of individual tweets). It is well-known that solving quantification by means of ``classify and count'' (i.e., by classifying all unlabelled items by means of a standard classifier and counting the items that have been assigned to a given class) is less than optimal in terms of accuracy, and that more accurate quantification methods exist. Gao and Sebastiani (2016) carried out a systematic comparison of quantification methods on the task of tweet sentiment quantification. In hindsight, we observe that the experimental protocol followed in that work was weak, and that the reliability of the conclusions that were drawn from the results is thus questionable. We now re-evaluate those quantification methods (plus a few more modern ones) on exactly the same same datasets, this time following a now consolidated and much more robust experimental protocol (which also involves simulating the presence, in the test data, of class prevalence values very different from those of the training set). This experimental protocol (even without counting the newly added methods) involves a number of experiments 5,775 times larger than that of the original study. The results of our experiments are dramatically different from those obtained by Gao and Sebastiani, and they provide a different, much more solid understanding of the relative strengths and weaknesses of different sentiment quantification methods.