mlsanityData to 7 Oct 2026

Demographic Dialectal Variation in Social Media: A Case Study of African-American English

arXiv 1608.08868 ↗First posted 31 Aug 2016cs.CLcited by 194

Not on the map. Why?

Abstract (arXiv)

Though dialectal language is increasingly abundant on social media, few resources exist for developing NLP tools to handle such language. We conduct a case study of dialectal language in online conversational text by investigating African-American English (AAE) on Twitter. We propose a distantly supervised model to identify AAE-like language from demographics associated with geo-located messages, and we verify that this language follows well-known AAE linguistic phenomena. In addition, we analyze the quality of existing language identification and dependency parsing tools on AAE-like text, demonstrating that they perform poorly on such text compared to text associated with white speakers. We also provide an ensemble classifier for language identification which eliminates this disparity and release a new corpus of tweets containing AAE-like language.

Key facts. Cited by 194 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Aug 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 38; 2022: 47; 2023: 33; 2024: 33; 2025: 26; 2026: 17.

194 papers.

Sort:

By area:

  1. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  2. 2021Ethical and social risks of harm from Language Modelscited by 957
  3. 2023A Comprehensive Overview of Large Language Modelscited by 639
  4. 2024OLMo: Accelerating the Science of Language Modelscited by 599
  5. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 522
  6. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
  7. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  8. 2021Gradient Matching for Domain Generalizationcited by 258
  9. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
  10. 2022Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Modelscited by 187
  11. 2022Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlationscited by 171
  12. 2021Challenges in Detoxifying Language Modelscited by 169
38 more
  1. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 154
  2. 2022Diverse Weight Averaging for Out-of-Distribution Generalizationcited by 148
  3. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  4. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  5. 2021Challenges in Automated Debiasing for Toxic Language Detectioncited by 95
  6. 2022The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymizationcited by 94
  7. 2021On Measures of Biases and Harms in NLPcited by 81
  8. 2021A Survey of Race, Racism, and Anti-Racism in NLPcited by 74
  9. 2021Towards generalisable hate speech detection: a review on obstacles and solutionscited by 66
  10. 2021Examining and Combating Spurious Features under Distribution Shiftcited by 64
  11. 2022Contrastive Adapters for Foundation Model Group Robustnesscited by 62
  12. 2023Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmarkcited by 62
  13. 2022Handling Bias in Toxic Speech Detection: A Surveycited by 60
  14. 2021Broaden the Vision: Geo-Diverse Visual Commonsense Reasoningcited by 59
  15. 2021A Systematic Investigation of Commonsense Knowledge in Large Language Modelscited by 56
  16. 2023Towards Last-layer Retraining for Group Robustness with Fewer Annotationscited by 54
  17. 2023Paloma: A Benchmark for Evaluating Language Model Fitcited by 53
  18. 2022Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selectioncited by 52
  19. 2022Multi-VALUE: A Framework for Cross-Dialectal English NLPcited by 51
  20. 2022Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Modelscited by 49
  21. 2021Diverse Adversaries for Mitigating Bias in Trainingcited by 48
  22. 2024Representation Surgery: Theory and Practice of Affine Steeringcited by 47
  23. 2022VALUE: Understanding Dialect Disparity in NLUcited by 45
  24. 2022Probing Classifiers are Unreliable for Concept Removal and Detectioncited by 40
  25. 2024Natural Language Processing for Dialects of a Language: A Surveycited by 40
  26. 2021Combining Feature and Instance Attribution to Detect Artifactscited by 38
  27. 2021Disembodied Machine Learning: On the Illusion of Objectivity in NLPcited by 37
  28. 2021A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representationscited by 37
  29. 2023Evaluation of African American Language Bias in Natural Language Generationcited by 36
  30. 2021Balancing out Bias: Achieving Fairness Through Balanced Trainingcited by 34
  31. 2021Contrastive Learning for Fair Representationscited by 34
  32. 2022Gold Doesn't Always Glitter: Spectral Removal of Linear and Nonlinear Guarded Attribute Informationcited by 33
  33. 2023Foundation Metrics for Evaluating Effectiveness of Healthcare Conversations Powered by Generative AIcited by 32
  34. 2024Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compressioncited by 32
  35. 2025Position: Model Collapse Does Not Mean What You Thinkcited by 32
  36. 2021Modeling the Second Player in Distributionally Robust Optimizationcited by 29
  37. 2022Understanding Why Generalized Reweighting Does Not Improve Over ERMcited by 29
  38. 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 27

50 most cited of 194 All 194 (JSON)