# Manufactured Divisiveness: Decomposing the Hostile Content of Seven Social Media Influence Operations

arXiv: https://arxiv.org/abs/2607.14491 · HTML: https://mlsanity.com/paper/2607.14491/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 56 references that are in the map.
- First posted on arXiv 16 Jul 2026.

## Builds on (most cited first, 28 of 28)

- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models](https://mlsanity.com/paper/2308.01263/) (2023)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection](https://mlsanity.com/paper/2111.07997/) (2021)
- [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023)
- [Latent Hatred: A Benchmark for Understanding Implicit Hate Speech](https://mlsanity.com/paper/2109.05322/) (2021)
- [Directions in Abusive Language Training Data: Garbage In, Garbage Out](https://mlsanity.com/paper/2004.01670/) (2020)
- [The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation](https://mlsanity.com/paper/2211.02570/) (2022)
- [Understanding Abuse: A Typology of Abusive Language Detection Subtasks](https://mlsanity.com/paper/1705.09899/) (2017)
- [Analyzing the Digital Traces of Political Manipulation: The 2016 Russian Interference Twitter Campaign](https://mlsanity.com/paper/1802.04291/) (2018)
- [Bots increase exposure to negative and inflammatory content in online social systems](https://mlsanity.com/paper/1802.07292/) (2018)
- [From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models](https://mlsanity.com/paper/2305.17174/) (2023)
- [Disinformation Warfare: Understanding State-Sponsored Trolls on Twitter and Their Influence on the Web](https://mlsanity.com/paper/1801.09288/) (2018)
- [Keeping Humans in the Loop: Human-Centered Automated Annotation with Generative AI](https://mlsanity.com/paper/2409.09467/) (2024)
- ["Short is the Road that Leads from Fear to Hate": Fear Speech in Indian WhatsApp Groups](https://mlsanity.com/paper/2102.03870/) (2021)
- [Retweet-BERT: Political Leaning Detection Using Language Features and Information Diffusion on Social Networks](https://mlsanity.com/paper/2207.08349/) (2022)
- [Who Let The Trolls Out? Towards Understanding State-Sponsored Trolls](https://mlsanity.com/paper/1811.03130/) (2018)
- [Measuring bot and human behavioral dynamics](https://mlsanity.com/paper/1802.04286/) (2018)
- [Evidence of Inter-state Coordination amongst State-backed Information Operations](https://mlsanity.com/paper/2305.05907/) (2023)
- [Labeled Datasets for Research on Information Operations](https://mlsanity.com/paper/2411.10609/) (2024)
- [Unraveling the Web of Disinformation: Exploring the Larger Context of State-Sponsored Influence Campaigns on Twitter](https://mlsanity.com/paper/2407.18098/) (2024)
- [The Language of Influence: Sentiment, Emotion, and Hate Speech in State Sponsored Influence Operations](https://mlsanity.com/paper/2505.07212/) (2025)
- [Toxicity in State Sponsored Information Operations](https://mlsanity.com/paper/2507.10936/) (2025)
- [RLHF May Not Reflect Genuine Preferences](https://mlsanity.com/paper/2604.03238/) (2026)
- [Cultural Targets, Structural Frames, Binding Morals: A Cross-Lingual Audit of Online Hate in Multicultural Singapore](https://mlsanity.com/paper/2606.21996/) (2026)

## Abstract (arXiv)

State-backed influence operations are routinely measured as high-prevalence sources of ``hate'' and ``toxicity.'' We argue those rates rest on a measurement error: the detectors behind them are validated to catch a broader definition inclusive of hostility or divisiveness aimed at an out-group, and so over-attribute hate to content better described as partisan or geopolitical invective. Across 25.08M tweets from seven government-attributed campaigns in the Twitter Information Operations archive (8,275 accounts), we separate hate from the other forms of divisiveness. We first validate a two-prompt LLM-based detector, matching human labels at Cohen's kappa=0.82, to identify the broader hostility; we then develop an auditable rule, agreeing with an expert at kappa=0.52, to further classify this content (5,457 posts) into three sub-categories. About 50.1% are identity-based attacks on people, whereas 30.4% are partisan attacks and 19.5% invective against states and their foreign policy. Reporting all of it as hate therefore overstates hate roughly twofold; only 18.7% is both identity-based and dehumanizing or inciting. Six of seven campaigns sort into three regimes that a single ``hate'' rate flattens, namely identity hate (RU-op and IRA, both Russia-attributed), geopolitical invective (both Iran operations), and partisan divisiveness (both Venezuela operations). We call the shared product manufactured divisiveness. The line to separate these constructs itself remains unsettled: on the hardest cases three independent human experts agree only moderately (pairwise kappa=0.37--0.50), and the best of nineteen LLM models tops out at kappa=0.601 against the experts' majority. Our findings can help redefine the study of hate in the context of influence campaigns and broader online discourse.