# Using Personality Detection Tools for Software Engineering Research: How Far Can We Go?

arXiv: https://arxiv.org/abs/2110.05035 · HTML: https://mlsanity.com/paper/2110.05035/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 128 references that are in the map.
- First posted on arXiv 11 Oct 2021.

## Builds on (most cited first, 18 of 18)

- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Automatic Text-based Personality Recognition on Monologues and Multiparty Dialogues Using Attentive Networks and Contextual Embeddings](https://mlsanity.com/paper/1911.09304/) (2019)
- [Recent Trends in Deep Learning Based Personality Detection](https://mlsanity.com/paper/1908.03628/) (2019)
- [Personality Trait Detection Using Bagged SVM over BERT Word Embedding Ensembles](https://mlsanity.com/paper/2010.01309/) (2020)
- [Sentiment Polarity Detection for Software Development](https://mlsanity.com/paper/1709.02984/) (2017)
- [A Language-independent and Compositional Model for Personality Trait Recognition from Short Texts](https://mlsanity.com/paper/1610.04345/) (2016)
- [25 Tweets to Know You: A New Model to Predict Personality with Social Media](https://mlsanity.com/paper/1704.05513/) (2017)
- [The Impact of Human Factors on the Participation Decision of Reviewers in Modern Code Review](https://mlsanity.com/paper/1806.10277/) (2018)
- [A Gold Standard for Emotion Annotation in Stack Overflow](https://mlsanity.com/paper/1803.02300/) (2018)
- [Natural Language or Not (NLoN) - A Package for Software Engineering Text Analysis Pipeline](https://mlsanity.com/paper/1803.07292/) (2018)
- [A large-scale, in-depth analysis of developers' personalities in the Apache ecosystem](https://mlsanity.com/paper/1905.13062/) (2019)
- [A Preliminary Analysis on the Effects of Propensity to Trust in Distributed Software Development](https://mlsanity.com/paper/1702.04958/) (2017)
- [Inferred vs traditional personality assessment: are we predicting the same thing?](https://mlsanity.com/paper/2103.09632/) (2021)
- [Finding Relationships between Socio-Technical Aspects and Personality Traits by Mining Developer E-mails](https://mlsanity.com/paper/1603.00696/) (2016)
- [EMTk -- The Emotion Mining Toolkit](https://mlsanity.com/paper/1903.09525/) (2019)
- [The Connection Between Burnout and Personality Types in Software Developers](https://mlsanity.com/paper/1906.09463/) (2019)
- [Communication and Personality Profiles of Global Software Developers](https://mlsanity.com/paper/2102.12563/) (2021)
- [In Search of Socio-Technical Congruence: A Large-Scale Longitudinal Study](https://mlsanity.com/paper/2105.08198/) (2021)

## Abstract (arXiv)

Assessing the personality of software engineers may help to match individual traits with the characteristics of development activities such as code review and testing, as well as support managers in team composition. However, self-assessment questionnaires are not a practical solution for collecting multiple observations on a large scale. Instead, automatic personality detection, while overcoming these limitations, is based on off-the-shelf solutions trained on non-technical corpora, which might not be readily applicable to technical domains like Software Engineering (SE). In this paper, we first assess the performance of general-purpose personality detection tools when applied to a technical corpus of developers' emails retrieved from the public archives of the Apache Software Foundation. We observe a general low accuracy of predictions and an overall disagreement among the tools. Second, we replicate two previous research studies in SE by replacing the personality detection tool used to infer developers' personalities from pull-request discussions and emails. We observe that the original results are not confirmed, i.e., changing the tool used in the original study leads to diverging conclusions. Our results suggest a need for personality detection tools specially targeted for the software engineering domain.