# A Study on Bias Detection and Classification in Natural Language Processing

arXiv: https://arxiv.org/abs/2408.07479 · HTML: https://mlsanity.com/paper/2408.07479/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 45 references that are in the map.
- First posted on arXiv 14 Aug 2024.

## Builds on (most cited first, 25 of 25)

- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems](https://mlsanity.com/paper/1805.04508/) (2018)
- [Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection](https://mlsanity.com/paper/2012.15761/) (2020)
- [Deep Learning for Hate Speech Detection in Tweets](https://mlsanity.com/paper/1706.00188/) (2017)
- [A Categorical Archive of ChatGPT Failures](https://mlsanity.com/paper/2302.03494/) (2023)
- [Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases](https://mlsanity.com/paper/2006.03955/) (2020)
- [Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation](https://mlsanity.com/paper/1911.03842/) (2019)
- [RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models](https://mlsanity.com/paper/2106.03521/) (2021)
- [Assessing Social and Intersectional Biases in Contextualized Word Representations](https://mlsanity.com/paper/1911.01485/) (2019)
- [Multilingual and Multi-Aspect Hate Speech Analysis](https://mlsanity.com/paper/1908.11049/) (2019)
- [CONAN -- COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech](https://mlsanity.com/paper/1910.03270/) (2019)
- [Evaluating the Underlying Gender Bias in Contextualized Word Embeddings](https://mlsanity.com/paper/1904.08783/) (2019)
- [What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus](https://mlsanity.com/paper/2105.02732/) (2021)
- [A Survey of Race, Racism, and Anti-Racism in NLP](https://mlsanity.com/paper/2106.11410/) (2021)
- [Gender-preserving Debiasing for Pre-trained Word Embeddings](https://mlsanity.com/paper/1906.00742/) (2019)
- [Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning](https://mlsanity.com/paper/2009.13028/) (2020)
- [Examining Gender Bias in Languages with Grammatical Gender](https://mlsanity.com/paper/1909.02224/) (2019)
- [Multi-label Categorization of Accounts of Sexism using a Neural Framework](https://mlsanity.com/paper/1910.04602/) (2019)
- [Multilingual Contextual Affective Analysis of LGBT People Portrayals in Wikipedia](https://mlsanity.com/paper/2010.10820/) (2020)
- [Towards Equal Gender Representation in the Annotations of Toxic Language Detection](https://mlsanity.com/paper/2106.02183/) (2021)

## Abstract (arXiv)

Human biases have been shown to influence the performance of models and algorithms in various fields, including Natural Language Processing. While the study of this phenomenon is garnering focus in recent years, the available resources are still relatively scarce, often focusing on different forms or manifestations of biases. The aim of our work is twofold: 1) gather publicly-available datasets and determine how to better combine them to effectively train models in the task of hate speech detection and classification; 2) analyse the main issues with these datasets, such as scarcity, skewed resources, and reliance on non-persistent data. We discuss these issues in tandem with the development of our experiments, in which we show that the combinations of different datasets greatly impact the models' performance.