# Investigating African-American Vernacular English in Transformer-Based Text Generation

arXiv: https://arxiv.org/abs/2010.02510 · HTML: https://mlsanity.com/paper/2010.02510/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2020.

## Built on by (most cited first, 30 of 53)

- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark](https://mlsanity.com/paper/2310.18018/) (2023)
- [Societal Biases in Language Generation: Progress and Challenges](https://mlsanity.com/paper/2105.04054/) (2021)
- [Automatically Auditing Large Language Models via Discrete Optimization](https://mlsanity.com/paper/2303.04381/) (2023)
- [Detoxifying Language Models Risks Marginalizing Minority Voices](https://mlsanity.com/paper/2104.06390/) (2021)
- [Capturing Failures of Large Language Models via Human Cognitive Biases](https://mlsanity.com/paper/2202.12299/) (2022)
- [Challenges in Automated Debiasing for Toxic Language Detection](https://mlsanity.com/paper/2102.00086/) (2021)
- [NLPositionality: Characterizing Design Biases of Datasets and Models](https://mlsanity.com/paper/2306.01943/) (2023)
- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)
- [On Measures of Biases and Harms in NLP](https://mlsanity.com/paper/2108.03362/) (2021)
- [A Survey of Race, Racism, and Anti-Racism in NLP](https://mlsanity.com/paper/2106.11410/) (2021)
- [Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models](https://mlsanity.com/paper/2206.08325/) (2022)
- [VALUE: Understanding Dialect Disparity in NLU](https://mlsanity.com/paper/2204.03031/) (2022)
- [Lost in Simulation: LLM-Simulated Users are Unreliable Proxies for Human Users in Agentic Evaluations](https://mlsanity.com/paper/2601.17087/) (2026)
- [Comparing Biases and the Impact of Multilingual Training across Multiple Languages](https://mlsanity.com/paper/2305.11242/) (2023)
- [Evaluation of African American Language Bias in Natural Language Generation](https://mlsanity.com/paper/2305.14291/) (2023)
- [Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks](https://mlsanity.com/paper/2502.13175/) (2025)
- [Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimization](https://mlsanity.com/paper/2210.04492/) (2022)
- [Learning Interpretable Style Embeddings via Prompting LLMs](https://mlsanity.com/paper/2305.12696/) (2023)
- [An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Models](https://mlsanity.com/paper/2301.09211/) (2023)
- [A Material Lens on Coloniality in NLP](https://mlsanity.com/paper/2311.08391/) (2023)
- [Rejected Dialects: Biases Against African American Language in Reward Models](https://mlsanity.com/paper/2502.12858/) (2025)
- [Investigating Memorization of Conspiracy Theories in Text Generation](https://mlsanity.com/paper/2101.00379/) (2021)
- [Language Detoxification with Attribute-Discriminative Latent Space](https://mlsanity.com/paper/2210.10329/) (2022)
- ["One-Size-Fits-All"? Examining Expectations around What Constitute "Fair" or "Good" NLG System Behaviors](https://mlsanity.com/paper/2310.15398/) (2023)
- [This Prompt is Measuring <MASK>: Evaluating Bias Evaluation in Language Models](https://mlsanity.com/paper/2305.12757/) (2023)
- [HERB: Measuring Hierarchical Regional Bias in Pre-trained Language Models](https://mlsanity.com/paper/2211.02882/) (2022)
- [Translational NLP: A New Paradigm and General Principles for Natural Language Processing Research](https://mlsanity.com/paper/2104.07874/) (2021)
- [Analyzing Hate Speech Data along Racial, Gender and Intersectional Axes](https://mlsanity.com/paper/2205.06621/) (2022)
- [Unsupervised Concept Vector Extraction for Bias Control in LLMs](https://mlsanity.com/paper/2502.19721/) (2025)

## Abstract (arXiv)

The growth of social media has encouraged the written use of African American Vernacular English (AAVE), which has traditionally been used only in oral contexts. However, NLP models have historically been developed using dominant English varieties, such as Standard American English (SAE), due to text corpora availability. We investigate the performance of GPT-2 on AAVE text by creating a dataset of intent-equivalent parallel AAVE/SAE tweet pairs, thereby isolating syntactic structure and AAVE- or SAE-specific language for each pair. We evaluate each sample and its GPT-2 generated text with pretrained sentiment classifiers and find that while AAVE text results in more classifications of negative sentiment than SAE, the use of GPT-2 generally increases occurrences of positive sentiment for both. Additionally, we conduct human evaluation of AAVE and SAE text generated with GPT-2 to compare contextual rigor and overall quality.