# RECAST: Enabling User Recourse and Interpretability of Toxicity Detection Models with Interactive Visualization

arXiv: https://arxiv.org/abs/2102.04427 · HTML: https://mlsanity.com/paper/2102.04427/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 64 references that are in the map.
- First posted on arXiv 8 Feb 2021.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [Is Attention Interpretable?](https://mlsanity.com/paper/1906.03731/) (2019)
- [TextBugger: Generating Adversarial Text Against Real-world Applications](https://mlsanity.com/paper/1812.05271/) (2018)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings](https://mlsanity.com/paper/1904.04047/) (2019)
- [Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers](https://mlsanity.com/paper/1801.06889/) (2018)
- [Automatically Neutralizing Subjective Bias in Text](https://mlsanity.com/paper/1911.09709/) (2019)
- [Deceiving Google's Perspective API Built for Detecting Toxic Comments](https://mlsanity.com/paper/1702.08138/) (2017)
- ["Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans](https://mlsanity.com/paper/2001.05871/) (2020)
- [Thou shalt not hate: Countering Online Hate Speech](https://mlsanity.com/paper/1808.04409/) (2018)
- [exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers Models](https://mlsanity.com/paper/1910.05276/) (2019)
- [Mean Birds: Detecting Aggression and Bullying on Twitter](https://mlsanity.com/paper/1702.06877/) (2017)
- [SANVis: Visual Analytics for Understanding Self-Attention Networks](https://mlsanity.com/paper/1909.09595/) (2019)
- [Neural Based Statement Classification for Biased Language](https://mlsanity.com/paper/1811.05740/) (2018)
- [A Visual Analytics Framework for Adversarial Text Generation](https://mlsanity.com/paper/1909.11202/) (2019)

## Built on by (most cited first, 6 of 6)

- [ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writing](https://mlsanity.com/paper/2305.09770/) (2023)
- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [DeMod: A Holistic Tool with Explainable Detection and Personalized Modification for Toxicity Censorship](https://mlsanity.com/paper/2411.01844/) (2024)
- [AI and the Future of Digital Public Squares](https://mlsanity.com/paper/2412.09988/) (2024)
- [Emotionally Aware Moderation: The Potential of Emotion Monitoring in Shaping Healthier Social Media Conversations](https://mlsanity.com/paper/2507.21089/) (2025)
- [WATCHED: A Web AI Agent Tool for Combating Hate Speech by Expanding Data](https://mlsanity.com/paper/2509.01379/) (2025)

## Abstract (arXiv)

With the widespread use of toxic language online, platforms are increasingly using automated systems that leverage advances in natural language processing to automatically flag and remove toxic comments. However, most automated systems -- when detecting and moderating toxic language -- do not provide feedback to their users, let alone provide an avenue of recourse for these users to make actionable changes. We present our work, RECAST, an interactive, open-sourced web tool for visualizing these models' toxic predictions, while providing alternative suggestions for flagged toxic language. Our work also provides users with a new path of recourse when using these automated moderation tools. RECAST highlights text responsible for classifying toxicity, and allows users to interactively substitute potentially toxic phrases with neutral alternatives. We examined the effect of RECAST via two large-scale user evaluations, and found that RECAST was highly effective at helping users reduce toxicity as detected through the model. Users also gained a stronger understanding of the underlying toxicity criterion used by black-box models, enabling transparency and recourse. In addition, we found that when users focus on optimizing language for these models instead of their own judgement (which is the implied incentive and goal of deploying automated models), these models cease to be effective classifiers of toxicity compared to human annotations. This opens a discussion for how toxicity detection models work and should work, and their effect on the future of online discourse.