CEA-LIST at CheckThat! 2025: Evaluating LLMs as Detectors of Bias and Opinion in Text
Not on the map. Why?
Abstract (arXiv)
This paper presents a competitive approach to multilingual subjectivity detection using large language models (LLMs) with few-shot prompting. We participated in Task 1: Subjectivity of the CheckThat! 2025 evaluation campaign. We show that LLMs, when paired with carefully designed prompts, can match or outperform fine-tuned smaller language models (SLMs), particularly in noisy or low-quality data settings. Despite experimenting with advanced prompt engineering techniques, such as debating LLMs and various example selection strategies, we found limited benefit beyond well-crafted standard few-shot prompts. Our system achieved top rankings across multiple languages in the CheckThat! 2025 subjectivity detection task, including first place in Arabic and Polish, and top-four finishes in Italian, English, German, and multilingual tracks. Notably, our method proved especially robust on the Arabic dataset, likely due to its resilience to annotation inconsistencies. These findings highlight the effectiveness and adaptability of LLM-based few-shot learning for multilingual sentiment tasks, offering a strong alternative to traditional fine-tuning, particularly when labeled data is scarce or inconsistent.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 15 of its 32 references that are in the map. First posted on arXiv 10 Jul 2025.
Builds on
15 of 32 references.
References · 15
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
- 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
- 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
- 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
- 2023Sentiment Analysis in the Era of Large Language Models: A Reality Checkcited by 262
- 2024ProSA: Assessing and Understanding the Prompt Sensitivity of LLMscited by 145
- 2024What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineeringcited by 88
- 2023kNN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inferencecited by 47
- 2018Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthinesscited by 41
- 2019A Transformer-based approach to Irony and Sarcasm detectioncited by 28
Built on by
Citing papers per year: 2026: 1.
1 paper.