mlsanityData to 7 Oct 2026

CEA-LIST at CheckThat! 2025: Evaluating LLMs as Detectors of Bias and Opinion in Text

arXiv 2507.07539 ↗First posted 10 Jul 2025cs.CL cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper presents a competitive approach to multilingual subjectivity detection using large language models (LLMs) with few-shot prompting. We participated in Task 1: Subjectivity of the CheckThat! 2025 evaluation campaign. We show that LLMs, when paired with carefully designed prompts, can match or outperform fine-tuned smaller language models (SLMs), particularly in noisy or low-quality data settings. Despite experimenting with advanced prompt engineering techniques, such as debating LLMs and various example selection strategies, we found limited benefit beyond well-crafted standard few-shot prompts. Our system achieved top rankings across multiple languages in the CheckThat! 2025 subjectivity detection task, including first place in Arabic and Polish, and top-four finishes in Italian, English, German, and multilingual tracks. Notably, our method proved especially robust on the Arabic dataset, likely due to its resilience to annotation inconsistencies. These findings highlight the effectiveness and adaptability of LLM-based few-shot learning for multilingual sentiment tasks, offering a strong alternative to traditional fine-tuning, particularly when labeled data is scarce or inconsistent.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 15 of its 32 references that are in the map. First posted on arXiv 10 Jul 2025.

Builds on

15 of 32 references.

References · 15

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  3. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  4. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  5. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  6. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
  7. 2023Sentiment Analysis in the Era of Large Language Models: A Reality Checkcited by 262
  8. 2024ProSA: Assessing and Understanding the Prompt Sensitivity of LLMscited by 145
  9. 2024What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineeringcited by 88
  10. 2023kNN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inferencecited by 47
  11. 2018Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthinesscited by 41
  12. 2019A Transformer-based approach to Irony and Sarcasm detectioncited by 28
3 more
  1. 2025The CLEF-2025 CheckThat! Lab: Subjectivity, Fact-Checking, Claim Normalization, and Retrievalcited by 14
  2. 2021Lexicon-based Methods vs. BERT for Text Sentiment Analysiscited by 4
  3. 2025Structured Reasoning for Fairness: A Multi-Agent Approach to Bias Detection in Textual Datacited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.