# Exposing propaganda: an analysis of stylistic cues comparing human annotations and machine classification

arXiv: https://arxiv.org/abs/2402.03780 · HTML: https://mlsanity.com/paper/2402.03780/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 20 references that are in the map.
- First posted on arXiv 6 Feb 2024.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [CamemBERT: a Tasty French Language Model](https://mlsanity.com/paper/1911.03894/) (2019)
- [SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles](https://mlsanity.com/paper/2009.02696/) (2020)
- [A Survey on Computational Propaganda Detection](https://mlsanity.com/paper/2007.08024/) (2020)
- [This Just In: Fake News Packs a Lot in Title, Uses Simpler, Repetitive Content in Text Body, More Similar to Satire than Real News](https://mlsanity.com/paper/1703.09398/) (2017)
- [Combining Vagueness Detection with Deep Learning to Identify Fake News](https://mlsanity.com/paper/2110.14780/) (2021)
- [Measuring vagueness and subjectivity in texts: from symbolic to neural VAGO](https://mlsanity.com/paper/2309.06132/) (2023)

## Built on by (most cited first, 4 of 4)

- [A Multi-Label Dataset of French Fake News: Human and Machine Insights](https://mlsanity.com/paper/2403.16099/) (2024)
- [Embedding Style Beyond Topics: Analyzing Dispersion Effects Across Different Language Models](https://mlsanity.com/paper/2501.00828/) (2025)
- [Reliable News or Propagandist News? A Neurosymbolic Model Using Genre, Topic, and Persuasion Techniques to Improve Robustness in Classification](https://mlsanity.com/paper/2604.01936/) (2026)
- [Propaganda Forensics: Recovering the Generation Pipeline of an AI-Driven Influence Campaign](https://mlsanity.com/paper/2608.15746/) (2026)

## Abstract (arXiv)

This paper investigates the language of propaganda and its stylistic features. It presents the PPN dataset, standing for Propagandist Pseudo-News, a multisource, multilingual, multimodal dataset composed of news articles extracted from websites identified as propaganda sources by expert agencies. A limited sample from this set was randomly mixed with papers from the regular French press, and their URL masked, to conduct an annotation-experiment by humans, using 11 distinct labels. The results show that human annotators were able to reliably discriminate between the two types of press across each of the labels. We propose different NLP techniques to identify the cues used by the annotators, and to compare them with machine classification. They include the analyzer VAGO to measure discourse vagueness and subjectivity, a TF-IDF to serve as a baseline, and four different classifiers: two RoBERTa-based models, CATS using syntax, and one XGBoost combining syntactic and semantic features.