# Beyond Dataset Creation: Critical View of Annotation Variation and Bias Probing of a Dataset for Online Radical Content Detection

arXiv: https://arxiv.org/abs/2412.11745 · HTML: https://mlsanity.com/paper/2412.11745/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 90 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Builds on (most cited first, 30 of 35)

- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection](https://mlsanity.com/paper/2111.07997/) (2021)
- [PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits](https://mlsanity.com/paper/2305.02547/) (2023)
- [Quantifying the Persona Effect in LLM Simulations](https://mlsanity.com/paper/2402.10811/) (2024)
- [XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond](https://mlsanity.com/paper/2104.12250/) (2021)
- [When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Tasks](https://mlsanity.com/paper/2305.06626/) (2023)
- [Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP](https://mlsanity.com/paper/2005.14672/) (2020)
- [Exploring Cross-Cultural Differences in English Hate Speech Annotations: From Dataset Construction to Analysis](https://mlsanity.com/paper/2308.16705/) (2023)
- [VariErr NLI: Separating Annotation Error from Human Label Variation](https://mlsanity.com/paper/2403.01931/) (2024)
- [The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels](https://mlsanity.com/paper/2405.05860/) (2024)
- [Can Large Language Models Capture Dissenting Human Voices?](https://mlsanity.com/paper/2305.13788/) (2023)
- [Identifying Morality Frames in Political Tweets using Relational Learning](https://mlsanity.com/paper/2109.04535/) (2021)
- [Detecting White Supremacist Hate Speech using Domain Specific Word Embedding with Deep Learning and BERT](https://mlsanity.com/paper/2010.00357/) (2020)
- [Large Language Models as Subpopulation Representative Models: A Review](https://mlsanity.com/paper/2310.17888/) (2023)
- [Interpretable Multi-Modal Hate Speech Detection](https://mlsanity.com/paper/2103.01616/) (2021)
- [Multilingual Auxiliary Tasks Training: Bridging the Gap between Languages for Zero-Shot Transfer of Hate Speech Detection Models](https://mlsanity.com/paper/2210.13029/) (2022)
- [Wisdom of Instruction-Tuned Language Model Crowds. Exploring Model Label Variation](https://mlsanity.com/paper/2307.12973/) (2023)
- [Different Tastes of Entities: Investigating Human Label Variation in Named Entity Annotations](https://mlsanity.com/paper/2402.01423/) (2024)
- [Automatic Detection of Online Jihadist Hate Speech](https://mlsanity.com/paper/1803.04596/) (2018)
- [Understanding the Radical Mind: Identifying Signals to Detect Extremist Content on Twitter](https://mlsanity.com/paper/1905.08067/) (2019)
- [Assessing Generalization for Subpopulation Representative Modeling via In-Context Learning](https://mlsanity.com/paper/2402.07368/) (2024)
- [Establishing a New State-of-the-Art for French Named Entity Recognition](https://mlsanity.com/paper/2005.13236/) (2020)
- [A survey on extremism analysis using Natural Language Processing](https://mlsanity.com/paper/2104.04069/) (2021)

## Built on by (most cited first, 3 of 3)

- [IYKYK: Using language models to decode extremist cryptolects](https://mlsanity.com/paper/2506.05635/) (2025)
- [Mitigating Cross-Lingual Cultural Inconsistencies in LLMs via Consensus-Driven Preference Optimisation](https://mlsanity.com/paper/2605.12515/) (2026)
- [Semantically Orthogonal Framework for Citation Classification: Disentangling Intent and Content](https://mlsanity.com/paper/2601.05103/) (2026)

## Abstract (arXiv)

The proliferation of radical content on online platforms poses significant risks, including inciting violence and spreading extremist ideologies. Despite ongoing research, existing datasets and models often fail to address the complexities of multilingual and diverse data. To bridge this gap, we introduce a publicly available multilingual dataset annotated with radicalization levels, calls for action, and named entities in English, French, and Arabic. This dataset is pseudonymized to protect individual privacy while preserving contextual information. Beyond presenting our freely available dataset, we analyze the annotation process, highlighting biases and disagreements among annotators and their implications for model performance. Additionally, we use synthetic data to investigate the influence of socio-demographic traits on annotation patterns and model predictions. Our work offers a comprehensive examination of the challenges and opportunities in building robust datasets for radical content detection, emphasizing the importance of fairness and transparency in model development.