LLMs for Targeted Sentiment in News Headlines: Exploring the Descriptive-Prescriptive Dilemma
Not on the map. Why?
Abstract (arXiv)
News headlines often evoke sentiment by intentionally portraying entities in particular ways, making targeted sentiment analysis (TSA) of headlines a worthwhile but difficult task. Due to its subjectivity, creating TSA datasets can involve various annotation paradigms, from descriptive to prescriptive, either encouraging or limiting subjectivity. LLMs are a good fit for TSA due to their broad linguistic and world knowledge and in-context learning abilities, yet their performance depends on prompt design. In this paper, we compare the accuracy of state-of-the-art LLMs and fine-tuned encoder models for TSA of news headlines using descriptive and prescriptive datasets across several languages. Exploring the descriptive--prescriptive continuum, we analyze how performance is affected by prompt prescriptiveness, ranging from plain zero-shot to elaborate few-shot prompts. Finally, we evaluate the ability of LLMs to quantify uncertainty via calibration error and comparison to human label variation. We find that LLMs outperform fine-tuned encoders on descriptive datasets, while calibration and F1-score generally improve with increased prescriptiveness, yet the optimal level varies.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 21 of its 39 references that are in the map. First posted on arXiv 1 Mar 2024.
Builds on
21 of 39 references.
References · 21
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
- 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
- 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
- 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 299
- 2021Want To Reduce Labeling Cost? GPT-3 Can Helpcited by 196
- 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
- 2021MultiVerS: Improving scientific claim verification with weak supervision and full-document contextcited by 140
- 2023Spanish Pre-trained BERT Model and Evaluation Datacited by 81
- 2023Automated Annotation with Generative AI Requires Validationcited by 75
9 more
- 2022Data Curation Alone Can Stabilize In-context Learningcited by 58
- 2020Pre-training Polish Transformer-based Language Models at Scalecited by 21
- 2021BERTi'c -- The Transformer Language Model for Bosnian, Croatian, Montenegrin and Serbiancited by 21
- 2013Sentiment Analysis in the Newscited by 20
- 2020Context-Guided BERT for Targeted Aspect-Based Sentiment Analysiscited by 18
- 2016Stock trend prediction using news sentiment analysiscited by 9
- 2022A Dataset and BERT-based Models for Targeted Sentiment Analysis on Turkish Textscited by 7
- 2019Targeted Sentiment Analysis: A Data-Driven Categorizationcited by 3
- 2022A Spanish dataset for Targeted Sentiment Analysis of political headlinescited by 1
Built on by
Citing papers per year: 2024: 1; 2025: 0; 2026: 2.
3 papers.
- 2024A Scalable Data-Driven Framework for Systematic Analysis of SEC 10-K Filings Using Large Language Modelscited by 3
- 2026Enhancing Sentiment Classification and Irony Detection in Large Language Models through Advanced Prompt Engineering Techniquescited by 0
- 2026Consistency Analysis of Sentiment Predictions using Syntactic & Semantic Context Assessment Summarization (SSAS)cited by 0
- 2026Consistency Analysis of Sentiment Predictions using Syntactic & Semantic Context Assessment Summarization (SSAS)16 Apr 2026
- 2026Enhancing Sentiment Classification and Irony Detection in Large Language Models through Advanced Prompt Engineering Techniques13 Jan 2026
- 2024A Scalable Data-Driven Framework for Systematic Analysis of SEC 10-K Filings Using Large Language Models26 Sep 2024