# How AI Fails: An Interactive Pedagogical Tool for Demonstrating Dialectal Bias in Automated Toxicity Models

arXiv: https://arxiv.org/abs/2511.06676 · HTML: https://mlsanity.com/paper/2511.06676/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 22 references that are in the map.
- First posted on arXiv 10 Nov 2025.

## Builds on (most cited first, 8 of 8)

- [Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection](https://mlsanity.com/paper/2012.15761/) (2020)
- [Demographic Dialectal Variation in Social Media: A Case Study of African-American English](https://mlsanity.com/paper/1608.08868/) (2016)
- [The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models](https://mlsanity.com/paper/2008.05122/) (2020)
- [A Survey of Race, Racism, and Anti-Racism in NLP](https://mlsanity.com/paper/2106.11410/) (2021)
- [Algorithmic Arbitrariness in Content Moderation](https://mlsanity.com/paper/2402.16979/) (2024)
- [FairPy: A Toolkit for Evaluation of Prediction Biases and their Mitigation in Large Language Models](https://mlsanity.com/paper/2302.05508/) (2023)
- [ToxBuster: In-game Chat Toxicity Buster with BERT](https://mlsanity.com/paper/2305.12542/) (2023)
- [Investigating Label Bias and Representational Sources of Age-Related Disparities in Medical Segmentation](https://mlsanity.com/paper/2511.00477/) (2025)

## Abstract (arXiv)

Now that AI-driven moderation has become pervasive in everyday life, we often hear claims that "the AI is biased". While this is often said jokingly, the light-hearted remark reflects a deeper concern. How can we be certain that an online post flagged as "inappropriate" was not simply the victim of a biased algorithm? This paper investigates this problem using a dual approach. First, I conduct a quantitative benchmark of a widely used toxicity model (unitary/toxic-bert) to measure performance disparity between text in African-American English (AAE) and Standard American English (SAE). The benchmark reveals a clear, systematic bias: on average, the model scores AAE text as 1.8 times more toxic and 8.8 times higher for "identity hate". Second, I introduce an interactive pedagogical tool that makes these abstract biases tangible. The tool's core mechanic, a user-controlled "sensitivity threshold," demonstrates that the biased score itself is not the only harm; instead, the more-concerning harm is the human-set, seemingly neutral policy that ultimately operationalises discrimination. This work provides both statistical evidence of disparate impact and a public-facing tool designed to foster critical AI literacy.