# What is Sentiment Meant to Mean to Language Models?

arXiv: https://arxiv.org/abs/2405.02454 · HTML: https://mlsanity.com/paper/2405.02454/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 33 references that are in the map.
- First posted on arXiv 3 May 2024.

## Builds on (most cited first, 7 of 7)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SemEval-2017 Task 4: Sentiment Analysis in Twitter](https://mlsanity.com/paper/1912.00741/) (2019)
- [FinBERT: A Pretrained Language Model for Financial Communications](https://mlsanity.com/paper/2006.08097/) (2020)
- [Stance Detection on Social Media: State of the Art and Trends](https://mlsanity.com/paper/2006.03644/) (2020)
- [Stance Detection: A Practical Guide to Classifying Political Beliefs in Text](https://mlsanity.com/paper/2305.01723/) (2023)
- [Your Stance is Exposed! Analysing Possible Factors for Stance Detection on Social Media](https://mlsanity.com/paper/1908.03146/) (2019)
- [Evaluation is all you need. Prompting Generative Large Language Models for Annotation Tasks in the Social Sciences. A Primer using Open Models](https://mlsanity.com/paper/2401.00284/) (2023)

## Built on by (most cited first, 1 of 1)

- [Rapid Review of Generative AI in Smart Medical Applications](https://mlsanity.com/paper/2406.06627/) (2024)

## Abstract (arXiv)

Sentiment analysis is one of the most widely used techniques in text analysis. Recent advancements with Large Language Models have made it more accurate and accessible than ever, allowing researchers to classify text with only a plain English prompt. However, "sentiment" entails a wide variety of concepts depending on the domain and tools used. It has been used to mean emotion, opinions, market movements, or simply a general ``good-bad'' dimension. This raises a question: What exactly are language models doing when prompted to label documents by sentiment? This paper first overviews how sentiment is defined across different contexts, highlighting that it is a confounded measurement construct in that it entails multiple variables, such as emotional valence and opinion, without disentangling them. I then test three language models across two data sets with prompts requesting sentiment, valence, and stance classification. I find that sentiment labels most strongly correlate with valence labels. I further find that classification improves when researchers more precisely specify their dimension of interest rather than using the less well-defined concept of sentiment. I conclude by encouraging researchers to move beyond "sentiment" when feasible and use a more precise measurement construct.