mlsanityData to 7 Oct 2026

Language (Technology) is Power: A Critical Survey of "Bias" in NLP

arXiv 2005.14050 ↗First posted 28 May 2020cs.CL cs.CYcited by 951

Not on the map. Why?

Cited by 23 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We survey 146 papers analyzing "bias" in NLP systems, finding that their motivations are often vague, inconsistent, and lacking in normative reasoning, despite the fact that analyzing "bias" is an inherently normative process. We further find that these papers' proposed quantitative techniques for measuring or mitigating "bias" are poorly matched to their motivations and do not engage with the relevant literature outside of NLP. Based on these findings, we describe the beginnings of a path forward by proposing three recommendations that should guide work analyzing "bias" in NLP systems. These recommendations rest on a greater recognition of the relationships between language and social hierarchies, encouraging researchers and practitioners to articulate their conceptualizations of "bias"---i.e., what kinds of system behaviors are harmful, in what ways, to whom, and why, as well as the normative reasoning underlying these statements---and to center work around the lived experiences of members of communities affected by NLP systems, while interrogating and reimagining the power relations between technologists and such communities.

Key facts. Cited by 951 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 104; 2022: 132; 2023: 170; 2024: 203; 2025: 200; 2026: 142.

951 papers.

Sort:

By area:

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  4. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  5. 2023PaLM 2 Technical Reportcited by 1,154
  6. 2021Ethical and social risks of harm from Language Modelscited by 957
  7. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  8. 2022Galactica: A Large Language Model for Sciencecited by 736
  9. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  10. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  11. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  12. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 506
38 more
  1. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
  2. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  3. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  4. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  5. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
  6. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 288
  7. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
  8. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
  9. 2023Can Large Language Models Transform Computational Social Science?cited by 245
  10. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  11. 2021Latent Hatred: A Benchmark for Understanding Implicit Hate Speechcited by 216
  12. 2022Challenges and Strategies in Cross-Cultural NLPcited by 216
  13. 2021Societal Biases in Language Generation: Progress and Challengescited by 201
  14. 2021What Will it Take to Fix Benchmarking in Natural Language Understanding?cited by 200
  15. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 196
  16. 2023Gender bias and stereotypes in Large Language Modelscited by 185
  17. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 179
  18. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 177
  19. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 176
  20. 2021The Values Encoded in Machine Learning Researchcited by 172
  21. 2023The Capacity for Moral Self-Correction in Large Language Modelscited by 171
  22. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 166
  23. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 160
  24. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 154
  25. 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
  26. 2023Debiasing Vision-Language Models via Biased Promptscited by 147
  27. 2023Co-Writing with Opinionated Language Models Affects Users' Viewscited by 145
  28. 2023Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Modelscited by 144
  29. 2021Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Modelscited by 141
  30. 2021Can Machines Learn Morality? The Delphi Experimentcited by 140
  31. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 138
  32. 2023Auditing large language models: a three-layered approachcited by 135
  33. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 130
  34. 2023Large language models can accurately predict searcher preferencescited by 129
  35. 2021Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologiescited by 128
  36. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Modelscited by 128
  37. 2022Bias Mitigation for Machine Learning Classifiers: A Comprehensive Surveycited by 125
  38. 2021Gender Bias in Machine Translationcited by 116

50 most cited of 951 All 951 (JSON)