# LLMs for Targeted Sentiment in News Headlines: Exploring the Descriptive-Prescriptive Dilemma

arXiv: https://arxiv.org/abs/2403.00418 · HTML: https://mlsanity.com/paper/2403.00418/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 1 Mar 2024.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [State of What Art? A Call for Multi-Prompt LLM Evaluation](https://mlsanity.com/paper/2401.00595/) (2023)
- [Want To Reduce Labeling Cost? GPT-3 Can Help](https://mlsanity.com/paper/2108.13487/) (2021)
- [Reducing Sentiment Bias in Language Models via Counterfactual Evaluation](https://mlsanity.com/paper/1911.03064/) (2019)
- [MultiVerS: Improving scientific claim verification with weak supervision and full-document context](https://mlsanity.com/paper/2112.01640/) (2021)
- [Spanish Pre-trained BERT Model and Evaluation Data](https://mlsanity.com/paper/2308.02976/) (2023)
- [Automated Annotation with Generative AI Requires Validation](https://mlsanity.com/paper/2306.00176/) (2023)
- [Data Curation Alone Can Stabilize In-context Learning](https://mlsanity.com/paper/2212.10378/) (2022)
- [Pre-training Polish Transformer-based Language Models at Scale](https://mlsanity.com/paper/2006.04229/) (2020)
- [BERTi'c -- The Transformer Language Model for Bosnian, Croatian, Montenegrin and Serbian](https://mlsanity.com/paper/2104.09243/) (2021)
- [Sentiment Analysis in the News](https://mlsanity.com/paper/1309.6202/) (2013)
- [Context-Guided BERT for Targeted Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2010.07523/) (2020)
- [Stock trend prediction using news sentiment analysis](https://mlsanity.com/paper/1607.01958/) (2016)
- [A Dataset and BERT-based Models for Targeted Sentiment Analysis on Turkish Texts](https://mlsanity.com/paper/2205.04185/) (2022)
- [Targeted Sentiment Analysis: A Data-Driven Categorization](https://mlsanity.com/paper/1905.03423/) (2019)
- [A Spanish dataset for Targeted Sentiment Analysis of political headlines](https://mlsanity.com/paper/2208.13947/) (2022)

## Built on by (most cited first, 3 of 3)

- [A Scalable Data-Driven Framework for Systematic Analysis of SEC 10-K Filings Using Large Language Models](https://mlsanity.com/paper/2409.17581/) (2024)
- [Enhancing Sentiment Classification and Irony Detection in Large Language Models through Advanced Prompt Engineering Techniques](https://mlsanity.com/paper/2601.08302/) (2026)
- [Consistency Analysis of Sentiment Predictions using Syntactic & Semantic Context Assessment Summarization (SSAS)](https://mlsanity.com/paper/2604.15547/) (2026)

## Abstract (arXiv)

News headlines often evoke sentiment by intentionally portraying entities in particular ways, making targeted sentiment analysis (TSA) of headlines a worthwhile but difficult task. Due to its subjectivity, creating TSA datasets can involve various annotation paradigms, from descriptive to prescriptive, either encouraging or limiting subjectivity. LLMs are a good fit for TSA due to their broad linguistic and world knowledge and in-context learning abilities, yet their performance depends on prompt design. In this paper, we compare the accuracy of state-of-the-art LLMs and fine-tuned encoder models for TSA of news headlines using descriptive and prescriptive datasets across several languages. Exploring the descriptive--prescriptive continuum, we analyze how performance is affected by prompt prescriptiveness, ranging from plain zero-shot to elaborate few-shot prompts. Finally, we evaluate the ability of LLMs to quantify uncertainty via calibration error and comparison to human label variation. We find that LLMs outperform fine-tuned encoders on descriptive datasets, while calibration and F1-score generally improve with increased prescriptiveness, yet the optimal level varies.