# Close to Human-Level Agreement: Tracing Journeys of Violent Speech in Incel Posts with GPT-4-Enhanced Annotations

arXiv: https://arxiv.org/abs/2401.02001 · HTML: https://mlsanity.com/paper/2401.02001/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jan 2024.

## Built on by (most cited first, 7 of 7)

- [Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits](https://mlsanity.com/paper/2403.14791/) (2024)
- [A Community-Centric Perspective for Characterizing and Detecting Anti-Asian Violence-Provoking Speech](https://mlsanity.com/paper/2407.15227/) (2024)
- [Text Annotation via Inductive Coding: Comparing Human Experts to LLMs in Qualitative Data Analysis](https://mlsanity.com/paper/2512.00046/) (2025)
- [Investigating the Effects of LLM Use on Critical Thinking Under Time Constraints: Access Timing and Time Availability](https://mlsanity.com/paper/2603.08849/) (2026)
- [Digital Guardians: Can GPT-4, Perspective API, and Moderation API reliably detect hate speech in reader comments of German online newspapers?](https://mlsanity.com/paper/2501.01256/) (2025)
- [Legal Experts Disagree With Rationale Extraction Techniques for Explaining ECtHR Case Outcome Classification](https://mlsanity.com/paper/2601.12419/) (2026)
- [From Quotes to Concepts: Axial Coding of Political Debates with Ensemble LMs](https://mlsanity.com/paper/2601.15338/) (2026)

## Abstract (arXiv)

This study investigates the prevalence of violent language on incels.is. It evaluates GPT models (GPT-3.5 and GPT-4) for content analysis in social sciences, focusing on the impact of varying prompts and batch sizes on coding quality for the detection of violent speech. We scraped over 6.9M posts from incels.is and categorized a random sample into non-violent, explicitly violent, and implicitly violent content. Two human coders annotated 3,028 posts, which we used to tune and evaluate GPT-3.5 and GPT-4 models across different prompts and batch sizes regarding coding reliability. The best-performing GPT-4 model annotated an additional 30,000 posts for further analysis. Our findings indicate an overall increase in violent speech overtime on incels.is, both at the community and individual level, particularly among more engaged users. While directed violent language decreases, non-directed violent language increases, and self-harm content shows a decline, especially after 2.5 years of user activity. We find substantial agreement between both human coders (K = .65), while the best GPT-4 model yields good agreement with both human coders (K = 0.54 for Human A and K = 0.62 for Human B). Weighted and macro F1 scores further support this alignment. Overall, this research provides practical means for accurately identifying violent language at a large scale that can aid content moderation and facilitate next-step research into the causal mechanism and potential mitigations of violent expression and radicalization in communities like incels.is.