# Large Language Models for Propaganda Detection

arXiv: https://arxiv.org/abs/2310.06422 · HTML: https://mlsanity.com/paper/2310.06422/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 35 references that are in the map.
- First posted on arXiv 10 Oct 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Design Guidelines for Prompt Engineering Text-to-Image Generative Models](https://mlsanity.com/paper/2109.06977/) (2021)
- [Fine-Grained Analysis of Propaganda in News Articles](https://mlsanity.com/paper/1910.02517/) (2019)
- [SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles](https://mlsanity.com/paper/2009.02696/) (2020)
- [Zero-shot Text Classification With Generative Language Models](https://mlsanity.com/paper/1912.10165/) (2019)
- [Legal Prompt Engineering for Multilingual Legal Judgement Prediction](https://mlsanity.com/paper/2212.02199/) (2022)
- [HealthPrompt: A Zero-shot Learning Paradigm for Clinical Natural Language Processing](https://mlsanity.com/paper/2203.05061/) (2022)
- [Practical Text Classification With Large Pre-Trained Language Models](https://mlsanity.com/paper/1812.01207/) (2018)
- [Prta: A System to Support the Analysis of Propaganda Techniques in the News](https://mlsanity.com/paper/2005.05854/) (2020)
- [Detecting Fake News Using Machine Learning : A Systematic Literature Review](https://mlsanity.com/paper/2102.04458/) (2021)
- [Overview of the WANLP 2022 Shared Task on Propaganda Detection in Arabic](https://mlsanity.com/paper/2211.10057/) (2022)
- [Large Language Models in the Workplace: A Case Study on Prompt Engineering for Job Type Classification](https://mlsanity.com/paper/2303.07142/) (2023)

## Built on by (most cited first, 8 of 8)

- [Think Fast, Think Slow, Think Critical: Designing an Automated Propaganda Detection Tool](https://mlsanity.com/paper/2402.19135/) (2024)
- [From Voices to Validity: Leveraging Large Language Models (LLMs) for Textual Analysis of Policy Stakeholder Interviews](https://mlsanity.com/paper/2312.01202/) (2023)
- [Unveiling Narrative Reasoning Limits of Large Language Models with Trope in Movie Synopses](https://mlsanity.com/paper/2409.14324/) (2024)
- [LlamBERT: Large-scale low-cost data annotation in NLP](https://mlsanity.com/paper/2403.15938/) (2024)
- [Assessing Generative Language Models in Classification Tasks: Performance and Self-Evaluation Capabilities in the Environmental and Climate Change Domain](https://mlsanity.com/paper/2408.17362/) (2024)
- [Effective Yet Ephemeral Propaganda Defense: There Needs to Be More than One-Shot Inoculation to Enhance Critical Thinking](https://mlsanity.com/paper/2503.16497/) (2025)
- [VIGIL: An Extensible System for Real-Time Detection and Mitigation of Cognitive Bias Triggers](https://mlsanity.com/paper/2604.03261/) (2026)
- [Unmasking Propaganda: A Comparative Analysis of Masked and Causal Language Models](https://mlsanity.com/paper/2610.03077/) (2026)

## Abstract (arXiv)

The prevalence of propaganda in our digital society poses a challenge to societal harmony and the dissemination of truth. Detecting propaganda through NLP in text is challenging due to subtle manipulation techniques and contextual dependencies. To address this issue, we investigate the effectiveness of modern Large Language Models (LLMs) such as GPT-3 and GPT-4 for propaganda detection. We conduct experiments using the SemEval-2020 task 11 dataset, which features news articles labeled with 14 propaganda techniques as a multi-label classification problem. Five variations of GPT-3 and GPT-4 are employed, incorporating various prompt engineering and fine-tuning strategies across the different models. We evaluate the models' performance by assessing metrics such as F1 score, Precision, and Recall, comparing the results with the current state-of-the-art approach using RoBERTa. Our findings demonstrate that GPT-4 achieves comparable results to the current state-of-the-art. Further, this study analyzes the potential and challenges of LLMs in complex tasks like propaganda detection.