mlsanityData to 7 Oct 2026

LLMs for Targeted Sentiment in News Headlines: Exploring the Descriptive-Prescriptive Dilemma

arXiv 2403.00418 ↗First posted 1 Mar 2024cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

News headlines often evoke sentiment by intentionally portraying entities in particular ways, making targeted sentiment analysis (TSA) of headlines a worthwhile but difficult task. Due to its subjectivity, creating TSA datasets can involve various annotation paradigms, from descriptive to prescriptive, either encouraging or limiting subjectivity. LLMs are a good fit for TSA due to their broad linguistic and world knowledge and in-context learning abilities, yet their performance depends on prompt design. In this paper, we compare the accuracy of state-of-the-art LLMs and fine-tuned encoder models for TSA of news headlines using descriptive and prescriptive datasets across several languages. Exploring the descriptive--prescriptive continuum, we analyze how performance is affected by prompt prescriptiveness, ranging from plain zero-shot to elaborate few-shot prompts. Finally, we evaluate the ability of LLMs to quantify uncertainty via calibration error and comparison to human label variation. We find that LLMs outperform fine-tuned encoders on descriptive datasets, while calibration and F1-score generally improve with increased prescriptiveness, yet the optimal level varies.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 21 of its 39 references that are in the map. First posted on arXiv 1 Mar 2024.

Builds on

21 of 39 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  5. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  6. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  7. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 299
  8. 2021Want To Reduce Labeling Cost? GPT-3 Can Helpcited by 196
  9. 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
  10. 2021MultiVerS: Improving scientific claim verification with weak supervision and full-document contextcited by 140
  11. 2023Spanish Pre-trained BERT Model and Evaluation Datacited by 81
  12. 2023Automated Annotation with Generative AI Requires Validationcited by 75
9 more
  1. 2022Data Curation Alone Can Stabilize In-context Learningcited by 58
  2. 2020Pre-training Polish Transformer-based Language Models at Scalecited by 21
  3. 2021BERTi'c -- The Transformer Language Model for Bosnian, Croatian, Montenegrin and Serbiancited by 21
  4. 2013Sentiment Analysis in the Newscited by 20
  5. 2020Context-Guided BERT for Targeted Aspect-Based Sentiment Analysiscited by 18
  6. 2016Stock trend prediction using news sentiment analysiscited by 9
  7. 2022A Dataset and BERT-based Models for Targeted Sentiment Analysis on Turkish Textscited by 7
  8. 2019Targeted Sentiment Analysis: A Data-Driven Categorizationcited by 3
  9. 2022A Spanish dataset for Targeted Sentiment Analysis of political headlinescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 2.

3 papers.