# Quantifying How Hateful Communities Radicalize Online Users

arXiv: https://arxiv.org/abs/2209.08697 · HTML: https://mlsanity.com/paper/2209.08697/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 40 references that are in the map.
- First posted on arXiv 19 Sep 2022.

## Builds on (most cited first, 14 of 14)

- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [The Pushshift Reddit Dataset](https://mlsanity.com/paper/2001.08435/) (2020)
- [Hate Speech Dataset from a White Supremacy Forum](https://mlsanity.com/paper/1809.04444/) (2018)
- [A Benchmark Dataset for Learning to Intervene in Online Hate Speech](https://mlsanity.com/paper/1909.04251/) (2019)
- [Hate Lingo: A Target-based Linguistic Analysis of Hate Speech in Social Media](https://mlsanity.com/paper/1804.04257/) (2018)
- [Analyzing the Targets of Hate in Online Social Media](https://mlsanity.com/paper/1603.07709/) (2016)
- [Quarantined! Examining the Effects of a Community-Wide Moderation Intervention on Reddit](https://mlsanity.com/paper/2009.11483/) (2020)
- [The Web Centipede: Understanding How Web Communities Influence Each Other Through the Lens of Mainstream and Alternative News Sources](https://mlsanity.com/paper/1705.06947/) (2017)
- [A Practical Introduction to Regression Discontinuity Designs: Foundations](https://mlsanity.com/paper/1911.09511/) (2019)
- [When the Echo Chamber Shatters: Examining the Use of Community-Specific Language Post-Subreddit Ban](https://mlsanity.com/paper/2106.16207/) (2021)
- [Characterizing the Language of Online Communities and its Relation to Community Reception](https://mlsanity.com/paper/1609.04779/) (2016)
- [Hate is not Binary: Studying Abusive Behavior of #GamerGate on Twitter](https://mlsanity.com/paper/1705.03345/) (2017)
- [Are Proactive Interventions for Reddit Communities Feasible?](https://mlsanity.com/paper/2111.11019/) (2021)
- [Benefits and costs of matching prior to a Difference in Differences analysis when parallel trends does not hold](https://mlsanity.com/paper/2205.08644/) (2022)

## Built on by (most cited first, 2 of 2)

- [Stranger Danger! Cross-Community Interactions with Fringe Users Increase the Growth of Fringe Communities on Reddit](https://mlsanity.com/paper/2310.12186/) (2023)
- [Large Language Models Help Reveal Unhealthy Diet and Body Concerns in Online Eating Disorders Communities](https://mlsanity.com/paper/2401.09647/) (2024)

## Abstract (arXiv)

While online social media offers a way for ignored or stifled voices to be heard, it also allows users a platform to spread hateful speech. Such speech usually originates in fringe communities, yet it can spill over into mainstream channels. In this paper, we measure the impact of joining fringe hateful communities in terms of hate speech propagated to the rest of the social network. We leverage data from Reddit to assess the effect of joining one type of echo chamber: a digital community of like-minded users exhibiting hateful behavior. We measure members' usage of hate speech outside the studied community before and after they become active participants. Using Interrupted Time Series (ITS) analysis as a causal inference method, we gauge the spillover effect, in which hateful language from within a certain community can spread outside that community by using the level of out-of-community hate word usage as a proxy for learned hate. We investigate four different Reddit sub-communities (subreddits) covering three areas of hate speech: racism, misogyny and fat-shaming. In all three cases we find an increase in hate speech outside the originating community, implying that joining such community leads to a spread of hate speech throughout the platform. Moreover, users are found to pick up this new hateful speech for months after initially joining the community. We show that the harmful speech does not remain contained within the community. Our results provide new evidence of the harmful effects of echo chambers and the potential benefit of moderating them to reduce adoption of hateful speech.