# Can GPT-4 Identify Propaganda? Annotation and Detection of Propaganda Spans in News Articles

arXiv: https://arxiv.org/abs/2402.17478 · HTML: https://mlsanity.com/paper/2402.17478/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 47 references that are in the map.
- First posted on arXiv 27 Feb 2024.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [MEGA: Multilingual Evaluation of Generative AI](https://mlsanity.com/paper/2303.12528/) (2023)
- [Fine-Grained Analysis of Propaganda in News Articles](https://mlsanity.com/paper/1910.02517/) (2019)
- [A Survey on Computational Propaganda Detection](https://mlsanity.com/paper/2007.08024/) (2020)
- [Detecting and Understanding Harmful Memes: A Survey](https://mlsanity.com/paper/2205.04274/) (2022)
- [SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Images](https://mlsanity.com/paper/2105.09284/) (2021)
- [A Survey on Multimodal Disinformation Detection](https://mlsanity.com/paper/2103.12541/) (2021)
- [Fighting the COVID-19 Infodemic in Social Media: A Holistic Perspective and a Call to Arms](https://mlsanity.com/paper/2007.07996/) (2020)
- [Argotario: Computational Argumentation Meets Serious Games](https://mlsanity.com/paper/1707.06002/) (2017)
- [LLMeBench: A Flexible Framework for Accelerating LLMs Benchmarking](https://mlsanity.com/paper/2308.04945/) (2023)
- [ArAIEval Shared Task: Persuasion Techniques and Disinformation Detection in Arabic Text](https://mlsanity.com/paper/2311.03179/) (2023)
- [Overview of the WANLP 2022 Shared Task on Propaganda Detection in Arabic](https://mlsanity.com/paper/2211.10057/) (2022)
- [Proppy: A System to Unmask Propaganda in Online News](https://mlsanity.com/paper/1912.06810/) (2019)
- [Can The Crowd Identify Misinformation Objectively? The Effects of Judgment Scale and Assessor's Background](https://mlsanity.com/paper/2005.06915/) (2020)
- [QCRI at SemEval-2023 Task 3: News Genre, Framing and Persuasion Techniques Detection using Multilingual Models](https://mlsanity.com/paper/2305.03336/) (2023)

## Built on by (most cited first, 21 of 21)

- [Must Read: A Comprehensive Survey of Computational Persuasion](https://mlsanity.com/paper/2505.07775/) (2025)
- [ArAIEval Shared Task: Propagandistic Techniques Detection in Unimodal and Multimodal Arabic Content](https://mlsanity.com/paper/2407.04247/) (2024)
- [VLDBench Evaluating Multimodal Disinformation with Regulatory Alignment](https://mlsanity.com/paper/2502.11361/) (2025)
- [Propaganda to Hate: A Multimodal Analysis of Arabic Memes with Multi-Agent LLMs](https://mlsanity.com/paper/2409.07246/) (2024)
- [PropXplain: Can LLMs Enable Explainable Propaganda Detection?](https://mlsanity.com/paper/2502.16550/) (2025)
- [LlamaLens: Specialized Multilingual LLM for Analyzing News and Social Media Content](https://mlsanity.com/paper/2410.15308/) (2024)
- [ArMeme: Propagandistic Content in Arabic Memes](https://mlsanity.com/paper/2406.03916/) (2024)
- [Hybrid Annotation for Propaganda Detection: Integrating LLM Pre-Annotations with Human Intelligence](https://mlsanity.com/paper/2507.18343/) (2025)
- [Detecting Winning Arguments with Large Language Models and Persuasion Strategies](https://mlsanity.com/paper/2601.10660/) (2026)
- [Exploring the Human-LLM Synergy in Advancing Theory-driven Qualitative Analysis](https://mlsanity.com/paper/2405.05758/) (2024)
- [Nullpointer at ArAIEval Shared Task: Arabic Propagandist Technique Detection with Token-to-Word Mapping in Sequence Tagging](https://mlsanity.com/paper/2407.01360/) (2024)
- [Fine-grained Narrative Classification in Biased News Articles](https://mlsanity.com/paper/2512.03582/) (2025)
- [When Agents Persuade: Rhetoric Generation and Mitigation in LLMs](https://mlsanity.com/paper/2603.04636/) (2026)
- [MemeMind at ArAIEval Shared Task: Spotting Persuasive Spans in Arabic Text with Persuasion Techniques Identification](https://mlsanity.com/paper/2408.04540/) (2024)
- [Combating Misinformation in the Arab World: Challenges & Opportunities](https://mlsanity.com/paper/2506.05582/) (2025)
- [From Emotion to Expression: Theoretical Foundations and Resources for Fear Speech](https://mlsanity.com/paper/2601.17132/) (2026)
- [CritiSense: Critical Digital Literacy and Resilience Against Misinformation](https://mlsanity.com/paper/2603.16672/) (2026)
- [Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas](https://mlsanity.com/paper/2605.13663/) (2026)
- [AHA-Memes: A Fine-Grained Multimodal Benchmark for Understanding Hate in Arabic Memes](https://mlsanity.com/paper/2607.27393/) (2026)
- [ProBel: Propaganda Detection with Techniques, Spans, and Explanations](https://mlsanity.com/paper/2608.22388/) (2026)
- [SWARM: A Multilingual Human-Annotated Dataset for Russian Propaganda Detection in Search Engine Results](https://mlsanity.com/paper/2609.12653/) (2026)

## Abstract (arXiv)

The use of propaganda has spiked on mainstream and social media, aiming to manipulate or mislead users. While efforts to automatically detect propaganda techniques in textual, visual, or multimodal content have increased, most of them primarily focus on English content. The majority of the recent initiatives targeting medium to low-resource languages produced relatively small annotated datasets, with a skewed distribution, posing challenges for the development of sophisticated propaganda detection models. To address this challenge, we carefully develop the largest propaganda dataset to date, ArPro, comprised of 8K paragraphs from newspaper articles, labeled at the text span level following a taxonomy of 23 propagandistic techniques. Furthermore, our work offers the first attempt to understand the performance of large language models (LLMs), using GPT-4, for fine-grained propaganda detection from text. Results showed that GPT-4's performance degrades as the task moves from simply classifying a paragraph as propagandistic or not, to the fine-grained task of detecting propaganda techniques and their manifestation in text. Compared to models fine-tuned on the dataset for propaganda detection at different classification granularities, GPT-4 is still far behind. Finally, we evaluate GPT-4 on a dataset consisting of six other languages for span detection, and results suggest that the model struggles with the task across languages. Our dataset and resources will be released to the community.