# Sentiment Analysis in the Era of Large Language Models: A Reality Check

arXiv: https://arxiv.org/abs/2305.15005 · HTML: https://mlsanity.com/paper/2305.15005/

## Key facts

- Cited by 268 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 77 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 30 of 33)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Instruction Tuning with GPT-4](https://mlsanity.com/paper/2304.03277/) (2023)
- [TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification](https://mlsanity.com/paper/2010.12421/) (2020)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [True Few-Shot Learning with Language Models](https://mlsanity.com/paper/2105.11447/) (2021)
- [Towards Emotional Support Dialog Systems](https://mlsanity.com/paper/2106.01144/) (2021)
- [UL2: Unifying Language Learning Paradigms](https://mlsanity.com/paper/2205.05131/) (2022)
- [A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models](https://mlsanity.com/paper/2303.10420/) (2023)
- [SemEval-2017 Task 4: Sentiment Analysis in Twitter](https://mlsanity.com/paper/1912.00741/) (2019)
- [SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)](https://mlsanity.com/paper/1903.08983/) (2019)
- [Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT](https://mlsanity.com/paper/2302.10198/) (2023)
- [The Multilingual Amazon Reviews Corpus](https://mlsanity.com/paper/2010.02573/) (2020)
- [Aspect Sentiment Quad Prediction as Paraphrase Generation](https://mlsanity.com/paper/2110.00796/) (2021)
- [Is ChatGPT a Good Sentiment Analyzer? A Preliminary Study](https://mlsanity.com/paper/2304.04339/) (2023)
- [Position-Aware Tagging for Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2010.02609/) (2020)
- [Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research](https://mlsanity.com/paper/2005.00357/) (2020)
- [A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges](https://mlsanity.com/paper/2203.01054/) (2022)

## Built on by (most cited first, 30 of 268)

- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Multilingual Jailbreak Challenges in Large Language Models](https://mlsanity.com/paper/2310.06474/) (2023)
- [Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs](https://mlsanity.com/paper/2402.03927/) (2024)
- [Large Language Models in Finance: A Survey](https://mlsanity.com/paper/2311.10723/) (2023)
- [EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers](https://mlsanity.com/paper/2309.08532/) (2023)
- [Benchmark Data Contamination of Large Language Models: A Survey](https://mlsanity.com/paper/2406.04244/) (2024)
- [LLM Post-Training: A Deep Dive into Reasoning Large Language Models](https://mlsanity.com/paper/2502.21321/) (2025)
- [Large Language Models Meet NLP: A Survey](https://mlsanity.com/paper/2405.12819/) (2024)
- [Safeguarding Large Language Models: A Survey](https://mlsanity.com/paper/2406.02622/) (2024)
- [Unleashing the potential of prompt engineering for large language models](https://mlsanity.com/paper/2310.14735/) (2023)
- [Thread of Thought Unraveling Chaotic Contexts](https://mlsanity.com/paper/2311.08734/) (2023)
- [LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Play](https://mlsanity.com/paper/2405.06373/) (2024)
- [CyberMetric: A Benchmark Dataset based on Retrieval-Augmented Generation for Evaluating LLMs in Cybersecurity Knowledge](https://mlsanity.com/paper/2402.07688/) (2024)
- [Probing LLMs for hate speech detection: strengths and vulnerabilities](https://mlsanity.com/paper/2310.12860/) (2023)
- [Designing Heterogeneous LLM Agents for Financial Sentiment Analysis](https://mlsanity.com/paper/2401.05799/) (2024)
- [Harnessing the Power of Multiple Minds: Lessons Learned from LLM Routing](https://mlsanity.com/paper/2405.00467/) (2024)
- [HelloBench: Evaluating Long Text Generation Capabilities of Large Language Models](https://mlsanity.com/paper/2409.16191/) (2024)
- [Exploring the Potential of Large Language Models in Computational Argumentation](https://mlsanity.com/paper/2311.09022/) (2023)
- [What is the Role of Small Models in the LLM Era: A Survey](https://mlsanity.com/paper/2409.06857/) (2024)
- [TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop](https://mlsanity.com/paper/2503.01013/) (2025)
- [Learning to Generate Explainable Stock Predictions using Self-Reflective Large Language Models](https://mlsanity.com/paper/2402.03659/) (2024)
- [Large Language Models as Markov Chains](https://mlsanity.com/paper/2410.02724/) (2024)
- [TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reduction](https://mlsanity.com/paper/2310.15556/) (2023)
- [A Wide Evaluation of ChatGPT on Affective Computing Tasks](https://mlsanity.com/paper/2308.13911/) (2023)
- [A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time](https://mlsanity.com/paper/2309.07418/) (2023)
- [DialogueLLM: Context and Emotion Knowledge-Tuned Large Language Models for Emotion Recognition in Conversations](https://mlsanity.com/paper/2310.11374/) (2023)
- [MatryoshkaKV: Adaptive KV Compression via Trainable Orthogonal Projection](https://mlsanity.com/paper/2410.14731/) (2024)
- [Can Small Language Models be Good Reasoners for Sequential Recommendation?](https://mlsanity.com/paper/2403.04260/) (2024)
- [Context versus Prior Knowledge in Language Models](https://mlsanity.com/paper/2404.04633/) (2024)
- [FFT: Towards Harmlessness Evaluation and Analysis for LLMs with Factuality, Fairness, Toxicity](https://mlsanity.com/paper/2311.18580/) (2023)

## Abstract (arXiv)

Sentiment analysis (SA) has been a long-standing research area in natural language processing. It can offer rich insights into human sentiments and opinions and has thus seen considerable interest from both academia and industry. With the advent of large language models (LLMs) such as ChatGPT, there is a great potential for their employment on SA problems. However, the extent to which existing LLMs can be leveraged for different sentiment analysis tasks remains unclear. This paper aims to provide a comprehensive investigation into the capabilities of LLMs in performing various sentiment analysis tasks, from conventional sentiment classification to aspect-based sentiment analysis and multifaceted analysis of subjective texts. We evaluate performance across 13 tasks on 26 datasets and compare the results against small language models (SLMs) trained on domain-specific datasets. Our study reveals that while LLMs demonstrate satisfactory performance in simpler tasks, they lag behind in more complex tasks requiring deeper understanding or structured sentiment information. However, LLMs significantly outperform SLMs in few-shot learning settings, suggesting their potential when annotation resources are limited. We also highlight the limitations of current evaluation practices in assessing LLMs' SA abilities and propose a novel benchmark, textscSentiEval, for a more comprehensive and realistic evaluation. Data and code during our investigations are available at urlhttps://github.com/DAMO-NLP-SG/LLM-Sentiment.