# Countering hate on social media: Large scale classification of hate and counter speech

arXiv: https://arxiv.org/abs/2006.01974 · HTML: https://mlsanity.com/paper/2006.01974/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2020.

## Built on by (most cited first, 22 of 22)

- [Hate Speech and Counter Speech Detection: Conversational Context Does Matter](https://mlsanity.com/paper/2206.06423/) (2022)
- [Beyond Denouncing Hate: Strategies for Countering Implied Biases and Stereotypes in Language](https://mlsanity.com/paper/2311.00161/) (2023)
- [NLP for Counterspeech against Hate: A Survey and How-To Guide](https://mlsanity.com/paper/2403.20103/) (2024)
- [Dataset for Identification of Homophobia and Transophobia in Multilingual YouTube Comments](https://mlsanity.com/paper/2109.00227/) (2021)
- [NLP Systems That Can't Tell Use from Mention Censor Counterspeech, but Teaching the Distinction Helps](https://mlsanity.com/paper/2404.01651/) (2024)
- [Understanding Counterspeech for Online Harm Mitigation](https://mlsanity.com/paper/2307.04761/) (2023)
- [Countering Online Hate Speech: An NLP Perspective](https://mlsanity.com/paper/2109.02941/) (2021)
- [HateRephrase: Zero- and Few-Shot Reduction of Hate Intensity in Online Posts using Large Language Models](https://mlsanity.com/paper/2310.13985/) (2023)
- [Counter Hate Speech in Social Media: A Survey](https://mlsanity.com/paper/2203.03584/) (2022)
- [Factoring Hate Speech: A New Annotation Framework to Study Hate Speech in Social Media](https://mlsanity.com/paper/2311.03969/) (2023)
- [Distilling Knowledge from Large Language Models: A Concept Bottleneck Model for Hate and Counter Speech Recognition](https://mlsanity.com/paper/2508.08274/) (2025)
- [TweetBLM: A Hate Speech Dataset and Analysis of Black Lives Matter-related Microblogs on Twitter](https://mlsanity.com/paper/2108.12521/) (2021)
- [Harnessing Artificial Intelligence to Combat Online Hate: Exploring the Challenges and Opportunities of Large Language Models in Hate Speech Detection](https://mlsanity.com/paper/2403.08035/) (2024)
- [Toxic language detection: a systematic review of Arabic datasets](https://mlsanity.com/paper/2312.07228/) (2023)
- [Hatred Stems from Ignorance! Distillation of the Persuasion Modes in Countering Conversational Hate Speech](https://mlsanity.com/paper/2403.15449/) (2024)
- [Effectiveness of Counter-Speech against Abusive Content: A Multidimensional Annotation and Classification Study](https://mlsanity.com/paper/2506.11919/) (2025)
- [Exploring the Plausibility of Hate and Counter Speech Detectors with Explainable AI](https://mlsanity.com/paper/2407.20274/) (2024)
- [Multi-dimensional Racism Classification during COVID-19: Stigmatization, Offensiveness, Blame, and Exclusion](https://mlsanity.com/paper/2208.13318/) (2022)
- [Designing Human-AI Collaboration to Support Learning in Counterspeech Writing](https://mlsanity.com/paper/2410.03032/) (2024)
- [Can NLP Tackle Hate Speech in the Real World? Stakeholder-Informed Feedback and Survey on Counterspeech](https://mlsanity.com/paper/2508.04638/) (2025)
- [Mapping Election Toxicity on Social Media across Issue, Ideology, and Psychosocial Dimensions](https://mlsanity.com/paper/2604.16765/) (2026)
- [Deconstructing Stereotypes: Scope-Conditioned Generation for Effective Multilingual Counterspeech](https://mlsanity.com/paper/2609.16906/) (2026)

## Abstract (arXiv)

Hateful rhetoric is plaguing online discourse, fostering extreme societal movements and possibly giving rise to real-world violence. A potential solution to this growing global problem is citizen-generated counter speech where citizens actively engage in hate-filled conversations to attempt to restore civil non-polarized discourse. However, its actual effectiveness in curbing the spread of hatred is unknown and hard to quantify. One major obstacle to researching this question is a lack of large labeled data sets for training automated classifiers to identify counter speech. Here we made use of a unique situation in Germany where self-labeling groups engaged in organized online hate and counter speech. We used an ensemble learning algorithm which pairs a variety of paragraph embeddings with regularized logistic regression functions to classify both hate and counter speech in a corpus of millions of relevant tweets from these two groups. Our pipeline achieved macro F1 scores on out of sample balanced test sets ranging from 0.76 to 0.97---accuracy in line and even exceeding the state of the art. On thousands of tweets, we used crowdsourcing to verify that the judgments made by the classifier are in close alignment with human judgment. We then used the classifier to discover hate and counter speech in more than 135,000 fully-resolved Twitter conversations occurring from 2013 to 2018 and study their frequency and interaction. Altogether, our results highlight the potential of automated methods to evaluate the impact of coordinated counter speech in stabilizing conversations on social media.