# Are Chess Discussions Racist? An Adversarial Hate Speech Data Set

arXiv: https://arxiv.org/abs/2011.10280 · HTML: https://mlsanity.com/paper/2011.10280/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2020.

## Built on by (most cited first, 10 of 10)

- [From Languages to Geographies: Towards Evaluating Cultural Bias in Hate Speech Datasets](https://mlsanity.com/paper/2404.17874/) (2024)
- [On the Calibration of Massively Multilingual Language Models](https://mlsanity.com/paper/2210.12265/) (2022)
- [Hate Speech Criteria: A Modular Approach to Task-Specific Hate Speech Definitions](https://mlsanity.com/paper/2206.15455/) (2022)
- [Vicarious Offense and Noise Audit of Offensive Speech Classifiers: Unifying Human and Machine Disagreement on What is Offensive](https://mlsanity.com/paper/2301.12534/) (2023)
- [Improving Adversarial Data Collection by Supporting Annotators: Lessons from GAHD, a German Hate Speech Dataset](https://mlsanity.com/paper/2403.19559/) (2024)
- ['Beach' to 'Bitch': Inadvertent Unsafe Transcription of Kids' Content on YouTube](https://mlsanity.com/paper/2203.04837/) (2022)
- [Auditing and Robustifying COVID-19 Misinformation Datasets via Anticontent Sampling](https://mlsanity.com/paper/2310.07078/) (2023)
- [LLMs and Finetuning: Benchmarking cross-domain performance for hate speech detection](https://mlsanity.com/paper/2310.18964/) (2023)
- [Towards Generalized Offensive Language Identification](https://mlsanity.com/paper/2407.18738/) (2024)
- [On the Challenges of Building Datasets for Hate Speech Detection](https://mlsanity.com/paper/2309.02912/) (2023)

## Abstract (arXiv)

On June 28, 2020, while presenting a chess podcast on Grandmaster Hikaru Nakamura, Antonio Radi'c's YouTube handle got blocked because it contained "harmful and dangerous" content. YouTube did not give further specific reason, and the channel got reinstated within 24 hours. However, Radi'c speculated that given the current political situation, a referral to "black against white", albeit in the context of chess, earned him this temporary ban. In this paper, via a substantial corpus of 681,995 comments, on 8,818 YouTube videos hosted by five highly popular chess-focused YouTube channels, we ask the following research question: how robust are off-the-shelf hate-speech classifiers to out-of-domain adversarial examples? We release a data set of 1,000 annotated comments where existing hate speech classifiers misclassified benign chess discussions as hate speech. We conclude with an intriguing analogy result on racial bias with our findings pointing out to the broader challenge of color polysemy.