# SWARM: A Multilingual Human-Annotated Dataset for Russian Propaganda Detection in Search Engine Results

arXiv: https://arxiv.org/abs/2609.12653 · HTML: https://mlsanity.com/paper/2609.12653/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 71 references that are in the map.
- First posted on arXiv 11 Sep 2026.

## Builds on (most cited first, 15 of 15)

- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [MEGA: Multilingual Evaluation of Generative AI](https://mlsanity.com/paper/2303.12528/) (2023)
- [Efficient Few-Shot Learning Without Prompts](https://mlsanity.com/paper/2209.11055/) (2022)
- [Fine-Grained Analysis of Propaganda in News Articles](https://mlsanity.com/paper/1910.02517/) (2019)
- [A Survey on Stance Detection for Mis- and Disinformation Identification](https://mlsanity.com/paper/2103.00242/) (2021)
- [A Survey on Computational Propaganda Detection](https://mlsanity.com/paper/2007.08024/) (2020)
- [Handling and Presenting Harmful Text in NLP Research](https://mlsanity.com/paper/2204.14256/) (2022)
- [Can GPT-4 Identify Propaganda? Annotation and Detection of Propaganda Spans in News Articles](https://mlsanity.com/paper/2402.17478/) (2024)
- [Russian propaganda on social media during the 2022 invasion of Ukraine](https://mlsanity.com/paper/2211.04154/) (2022)
- [EUvsDisinfo: A Dataset for Multilingual Detection of Pro-Kremlin Disinformation in News Articles](https://mlsanity.com/paper/2406.12614/) (2024)
- [Automated multilingual detection of Pro-Kremlin propaganda in newspapers and Telegram posts](https://mlsanity.com/paper/2301.10604/) (2023)
- [Cross-Domain Learning for Classifying Propaganda in Online Contents](https://mlsanity.com/paper/2011.06844/) (2020)
- [Neural Architectures for Fine-Grained Propaganda Detection in News](https://mlsanity.com/paper/1909.06162/) (2019)
- [Temporally Stable Multilayer Network Embeddings: A Longitudinal Study of Russian Propaganda](https://mlsanity.com/paper/2307.10264/) (2023)

## Abstract (arXiv)

Russian state propaganda spreads across many languages and online spaces. Yet, most computational work examines only one such space, usually social media, in one or two languages, and analyses sources rather than content. We introduce SWARM (Search-Web documents Annotated for Russian propaganda, Multilingual), a dataset of 2,183 search engine results across nine languages and diverse web domains (e.g., news, blogs, government sites), each annotated by trained coders for whether it supports a recurring Russian propaganda narrative. We benchmark a source-based blocklist, supervised classifiers, and zero-shot LLMs against these labels. The blocklist misses most propaganda-supporting documents, because such content is not confined to flagged "propaganda" outlets but also appears on mainstream ones. Content-level analysis helps, though how much depends on the model: the strongest LLM reaches a positive-class F1 of 0.73, whereas the supervised classifiers reach only about 0.5, with the smaller LLMs over-predicting support, mistaking topical relevance for endorsement. Detecting search-borne propaganda thus requires per-language, content-level evaluation, which we hope SWARM and our evaluation code enable.