# A Benchmark Study on Sentiment Analysis for Software Engineering Research

arXiv: https://arxiv.org/abs/1803.06525 · HTML: https://mlsanity.com/paper/1803.06525/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Mar 2018.

## Built on by (most cited first, 14 of 14)

- [A Comprehensive Empirical Study of Bias Mitigation Methods for Machine Learning Classifiers](https://mlsanity.com/paper/2207.03277/) (2022)
- [Seeking Common but Distinguishing Difference, A Joint Aspect-based Sentiment Analysis Model](https://mlsanity.com/paper/2111.09634/) (2021)
- [On the validity of pre-trained transformers for natural language processing in the software engineering domain](https://mlsanity.com/paper/2109.04738/) (2021)
- [BERT-Based Sentiment Analysis: A Software Engineering Perspective](https://mlsanity.com/paper/2106.02581/) (2021)
- [Automated Identification of Toxic Code Reviews Using ToxiCR](https://mlsanity.com/paper/2202.13056/) (2022)
- [Evaluating Pre-Trained Models for User Feedback Analysis in Software Engineering: A Study on Classification of App-Reviews](https://mlsanity.com/paper/2104.05861/) (2021)
- [Deep Learning-based Sentiment Classification: A Comparative Survey](https://mlsanity.com/paper/2312.17253/) (2023)
- [An Empirical Study of the Effectiveness of an Ensemble of Stand-alone Sentiment Detection Tools for Software Engineering Datasets](https://mlsanity.com/paper/2111.03196/) (2021)
- [TOUR: Dynamic Topic and Sentiment Analysis of User Reviews for Assisting App Release](https://mlsanity.com/paper/2103.15774/) (2021)
- [Polarity in the Classroom: A Case Study Leveraging Peer Sentiment Toward Scalable Assessment](https://mlsanity.com/paper/2108.10068/) (2021)
- [Do Not Take It for Granted: Comparing Open-Source Libraries for Software Development Effort Estimation](https://mlsanity.com/paper/2207.01705/) (2022)
- [Less, but Stronger: On the Value of Strong Heuristics in Semi-supervised Learning for Software Analytics](https://mlsanity.com/paper/2302.01997/) (2023)
- [Sentiment Analysis in Software Engineering: Evaluating Generative Pre-trained Transformers](https://mlsanity.com/paper/2505.14692/) (2025)
- [Sentiment analysis for software engineering: How far can zero-shot learning (ZSL) go?](https://mlsanity.com/paper/2604.13826/) (2026)

## Abstract (arXiv)

A recent research trend has emerged to identify developers' emotions, by applying sentiment analysis to the content of communication traces left in collaborative development environments. Trying to overcome the limitations posed by using off-the-shelf sentiment analysis tools, researchers recently started to develop their own tools for the software engineering domain. In this paper, we report a benchmark study to assess the performance and reliability of three sentiment analysis tools specifically customized for software engineering. Furthermore, we offer a reflection on the open challenges, as they emerge from a qualitative analysis of misclassified texts.