# Analysis of LLM as a grammatical feature tagger for African American English

arXiv: https://arxiv.org/abs/2502.06004 · HTML: https://mlsanity.com/paper/2502.06004/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 38 references that are in the map.
- First posted on arXiv 9 Feb 2025.

## Builds on (most cited first, 13 of 13)

- [Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems](https://mlsanity.com/paper/1805.04508/) (2018)
- [Demographic Dialectal Variation in Social Media: A Case Study of African-American English](https://mlsanity.com/paper/1608.08868/) (2016)
- [RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models](https://mlsanity.com/paper/2106.03521/) (2021)
- [Attention Sorting Combats Recency Bias In Long Context Language Models](https://mlsanity.com/paper/2310.01427/) (2023)
- [Multi-VALUE: A Framework for Cross-Dialectal English NLP](https://mlsanity.com/paper/2212.08011/) (2022)
- [Serial Position Effects of Large Language Models](https://mlsanity.com/paper/2406.15981/) (2024)
- [The Effectiveness of LLMs as Annotators: A Comparative Overview and Empirical Analysis of Direct Representation](https://mlsanity.com/paper/2405.01299/) (2024)
- [Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encoding](https://mlsanity.com/paper/2004.14870/) (2020)
- [Linear Recency Bias During Training Improves Transformers' Fit to Reading Times](https://mlsanity.com/paper/2409.11250/) (2024)
- [On Limitations of LLM as Annotator for Low Resource Languages](https://mlsanity.com/paper/2411.17637/) (2024)
- [Analyzing Hate Speech Data along Racial, Gender and Intersectional Axes](https://mlsanity.com/paper/2205.06621/) (2022)
- [Corpus-Guided Contrast Sets for Morphosyntactic Feature Detection in Low-Resource English Varieties](https://mlsanity.com/paper/2209.07611/) (2022)
- [Disambiguation of morpho-syntactic features of African American English -- the case of habitual be](https://mlsanity.com/paper/2204.12421/) (2022)

## Built on by (most cited first, 2 of 2)

- [Automatic Speech Recognition of African American English: Lexical and Contextual Effects](https://mlsanity.com/paper/2506.06888/) (2025)
- [A Sociolinguistic Analysis of Automatic Speech Recognition Bias in Newcastle English](https://mlsanity.com/paper/2603.24549/) (2026)

## Abstract (arXiv)

African American English (AAE) presents unique challenges in natural language processing (NLP). This research systematically compares the performance of available NLP models--rule-based, transformer-based, and large language models (LLMs)--capable of identifying key grammatical features of AAE, namely Habitual Be and Multiple Negation. These features were selected for their distinct grammatical complexity and frequency of occurrence. The evaluation involved sentence-level binary classification tasks, using both zero-shot and few-shot strategies. The analysis reveals that while LLMs show promise compared to the baseline, they are influenced by biases such as recency and unrelated features in the text such as formality. This study highlights the necessity for improved model training and architectural adjustments to better accommodate AAE's unique linguistic characteristics. Data and code are available.