# SCRum-9: Multilingual Stance Classification over Rumours on Social Media

arXiv: https://arxiv.org/abs/2505.18916 · HTML: https://mlsanity.com/paper/2505.18916/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 25 May 2025.

## Builds on (most cited first, 21 of 21)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Detection and Resolution of Rumours in Social Media: A Survey](https://mlsanity.com/paper/1704.00656/) (2017)
- [Don't Trust ChatGPT when Your Question is not in English: A Study of Multilingual Abilities and Types of LLMs](https://mlsanity.com/paper/2305.16339/) (2023)
- [SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours](https://mlsanity.com/paper/1704.05972/) (2017)
- [RumourEval 2019: Determining Rumour Veracity and Support for Rumours](https://mlsanity.com/paper/1809.06683/) (2018)
- [A Survey on Stance Detection for Mis- and Disinformation Identification](https://mlsanity.com/paper/2103.00242/) (2021)
- [X-Stance: A Multilingual Multi-Target Dataset for Stance Detection](https://mlsanity.com/paper/2003.08385/) (2020)
- [Few-Shot Cross-Lingual Stance Detection with Sentiment-Based Pre-Training](https://mlsanity.com/paper/2109.06050/) (2021)
- [CrowdTruth 2.0: Quality Metrics for Crowdsourcing with Disagreement](https://mlsanity.com/paper/1808.06080/) (2018)
- [Combating Disinformation in a Social Media Age](https://mlsanity.com/paper/2007.07388/) (2020)
- [Don't Waste a Single Annotation: Improving Single-Label Classifiers Through Soft Labels](https://mlsanity.com/paper/2311.05265/) (2023)
- [Multilingual Stance Detection: The Catalonia Independence Corpus](https://mlsanity.com/paper/2004.00050/) (2020)
- [VaxxHesitancy: A Dataset for Studying Hesitancy towards COVID-19 Vaccination on Twitter](https://mlsanity.com/paper/2301.06660/) (2023)
- [Multilingual Coarse Political Stance Classification of Media. The Editorial Line of a ChatGPT and Bard Newspaper](https://mlsanity.com/paper/2310.16269/) (2023)
- [Measuring What Counts: The case of Rumour Stance Classification](https://mlsanity.com/paper/2010.04532/) (2020)
- [Exploring Vision Language Models for Multimodal and Multilingual Stance Detection](https://mlsanity.com/paper/2501.17654/) (2025)
- [Stanceosaurus 2.0: Classifying Stance Towards Russian and Spanish Misinformation](https://mlsanity.com/paper/2402.03642/) (2024)
- [Can We Identify Stance Without Target Arguments? A Study for Rumour Stance Classification](https://mlsanity.com/paper/2303.12665/) (2023)

## Abstract (arXiv)

We introduce SCRum-9, the largest multilingual Stance Classification dataset for Rumour analysis in 9 languages, containing 7,516 tweets from X. SCRum-9 goes beyond existing stance classification datasets by covering more languages, linking examples to more fact-checked claims (2.1k), and including confidence-related annotations from multiple annotators to account for intra- and inter-annotator variability. Annotations were made by at least two native speakers per language, totalling more than 405 hours of annotation and 8,150 dollars in compensation. Further, SCRum-9 is used to benchmark five large language models (LLMs) and two multilingual masked language models (MLMs) in In-Context Learning (ICL) and fine-tuning setups. This paper also innovates by exploring the use of multilingual synthetic data for rumour stance classification, showing that even LLMs with weak ICL performance can produce valuable synthetic data for fine-tuning small MLMs, enabling them to achieve higher performance than zero-shot ICL in LLMs. Finally, we examine the relationship between model predictions and human uncertainty on ambiguous cases finding that model predictions often match the second-choice labels assigned by annotators, rather than diverging entirely from human judgments. SCRum-9 is publicly released to the research community with potential to foster further research on multilingual analysis of misleading narratives on social media.