# Are LLMs Enough for Hyperpartisan, Fake, Polarized and Harmful Content Detection? Evaluating In-Context Learning vs. Fine-Tuning

arXiv: https://arxiv.org/abs/2509.07768 · HTML: https://mlsanity.com/paper/2509.07768/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security.
- Builds on 30 of its 75 references that are in the map.
- First posted on arXiv 9 Sep 2025.

## Placed in

- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 30 of 30)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations](https://mlsanity.com/paper/2205.11822/) (2022)
- [The science of fake news](https://mlsanity.com/paper/2307.07903/) (2023)
- [Measuring Political Bias in Large Language Models: What Is Said and How It Is Said](https://mlsanity.com/paper/2403.18932/) (2024)
- [A Stylometric Inquiry into Hyperpartisan and Fake News](https://mlsanity.com/paper/1702.05638/) (2017)
- [The Turking Test: Can Language Models Understand Instructions?](https://mlsanity.com/paper/2010.11982/) (2020)
- [Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? A Study on Several Typical Tasks](https://mlsanity.com/paper/2305.05862/) (2023)
- [POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detection](https://mlsanity.com/paper/2205.00619/) (2022)
- [Disinformation Detection: An Evolving Challenge in the Age of LLMs](https://mlsanity.com/paper/2309.15847/) (2023)
- [Language Models for Text Classification: Is In-Context Learning Enough?](https://mlsanity.com/paper/2403.17661/) (2024)
- [HARE: Explainable Hate Speech Detection with Step-by-Step Reasoning](https://mlsanity.com/paper/2311.00321/) (2023)
- [Codebook LLMs: Evaluating LLMs as Measurement Tools for Political Science Concepts](https://mlsanity.com/paper/2407.10747/) (2024)
- [Diversity Measurement and Subset Selection for Instruction Tuning Datasets](https://mlsanity.com/paper/2402.02318/) (2024)
- [A Zero-shot and Few-shot Study of Instruction-Finetuned Large Language Models Applied to Clinical and Biomedical Tasks](https://mlsanity.com/paper/2307.12114/) (2023)
- [Monte Carlo with Determinantal Point Processes](https://mlsanity.com/paper/1605.00361/) (2016)
- [Parameter-Efficient Tuning Makes a Good Classification Head](https://mlsanity.com/paper/2210.16771/) (2022)
- [Weakly Supervised Veracity Classification with LLM-Predicted Credibility Signals](https://mlsanity.com/paper/2309.07601/) (2023)
- [Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought](https://mlsanity.com/paper/2404.03414/) (2024)
- [Effective Demonstration Annotation for In-Context Learning via Language Model-Based Determinantal Point Process](https://mlsanity.com/paper/2408.02103/) (2024)
- [Qbias -- A Dataset on Media Bias in Search Queries and Query Suggestions](https://mlsanity.com/paper/2311.17780/) (2023)
- [Are Large Language Models Good at Detecting Propaganda?](https://mlsanity.com/paper/2505.13706/) (2025)
- [P3: A Policy-Driven, Pace-Adaptive, and Diversity-Promoted Framework for data pruning in LLM Training](https://mlsanity.com/paper/2408.05541/) (2024)

## Built on by (most cited first, 1 of 1)

- [BITS Pilani at SemEval-2026 Task 9: Structured Supervised Fine-Tuning with DPO Refinement for Polarization Detection](https://mlsanity.com/paper/2604.11121/) (2026)

## Abstract (arXiv)

The spread of fake news, polarizing, politically biased, and harmful content on online platforms has been a serious concern. With large language models becoming a promising approach, however, no study has properly benchmarked their performance across different models, usage methods, and languages. This study presents a comprehensive overview of different Large Language Models adaptation paradigms for the detection of hyperpartisan and fake news, harmful tweets, and political bias. Our experiments spanned 10 datasets and 5 different languages (English, Spanish, Portuguese, Arabic and Bulgarian), covering both binary and multiclass classification scenarios. We tested different strategies ranging from parameter efficient Fine-Tuning of language models to a variety of different In-Context Learning strategies and prompts. These included zero-shot prompts, codebooks, few-shot (with both randomly-selected and diversely-selected examples using Determinantal Point Processes), and Chain-of-Thought. We discovered that In-Context Learning often underperforms when compared to Fine-Tuning a model. This main finding highlights the importance of Fine-Tuning even smaller models on task-specific settings even when compared to the largest models evaluated in an In-Context Learning setup - in our case LlaMA3.1-8b-Instruct, Mistral-Nemo-Instruct-2407 and Qwen2.5-7B-Instruct.