mlsanityData to 7 Oct 2026

CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models

arXiv 2010.00133 ↗First posted 30 Sep 2020cs.CL cs.AIcited by 703

Not on the map. Why?

Cited by 31 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Pretrained language models, especially masked language models (MLMs) have seen success across many NLP tasks. However, there is ample evidence that they use the cultural biases that are undoubtedly present in the corpora they are trained on, implicitly creating harm with biased representations. To measure some forms of social bias in language models against protected demographic groups in the US, we introduce the Crowdsourced Stereotype Pairs benchmark (CrowS-Pairs). CrowS-Pairs has 1508 examples that cover stereotypes dealing with nine types of bias, like race, religion, and age. In CrowS-Pairs a model is presented with two sentences: one that is more stereotyping and another that is less stereotyping. The data focuses on stereotypes about historically disadvantaged groups and contrasts them with advantaged groups. We find that all three of the widely-used MLMs we evaluate substantially favor sentences that express stereotypes in every category in CrowS-Pairs. As work on building less biased models advances, this dataset can be used as a benchmark to evaluate progress.

Key facts. Cited by 703 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 36; 2022: 65; 2023: 140; 2024: 175; 2025: 147; 2026: 140.

703 papers.

Sort:

By area:

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  4. 2023A Survey of Large Language Modelscited by 2,537
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  6. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  7. 2021Ethical and social risks of harm from Language Modelscited by 957
  8. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  9. 2023StarCoder: may the source be with you!cited by 790
  10. 2022Galactica: A Large Language Model for Sciencecited by 736
  11. 2023A Comprehensive Overview of Large Language Modelscited by 639
  12. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
38 more
  1. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  2. 2021MetaICL: Learning to Learn In Contextcited by 450
  3. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  4. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  5. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  6. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  7. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  8. 2022Prompting GPT-3 To Be Reliablecited by 258
  9. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
  10. 2023Large Language Model Alignment: A Surveycited by 241
  11. 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 235
  12. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 225
  13. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 204
  14. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
  15. 2021Efficient Large Scale Language Modeling with Mixtures of Expertscited by 175
  16. 2024Lessons from the Trenches on Reproducible Evaluation of Language Modelscited by 172
  17. 2023The Capacity for Moral Self-Correction in Large Language Modelscited by 171
  18. 2024BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languagescited by 171
  19. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 166
  20. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 160
  21. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  22. 2023Fundamental Limitations of Alignment in Large Language Modelscited by 152
  23. 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
  24. 2023Self-Alignment with Instruction Backtranslationcited by 149
  25. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 147
  26. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  27. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 138
  28. 2023Auditing large language models: a three-layered approachcited by 135
  29. 2021KLUE: Korean Language Understanding Evaluationcited by 113
  30. 2023Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendationcited by 113
  31. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 111
  32. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
  33. 2023Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Clustercited by 106
  34. 2021Sustainable Modular Debiasing of Language Modelscited by 98
  35. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 96
  36. 2024Representation Noising: A Defence Mechanism Against Harmful Finetuningcited by 93
  37. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  38. 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 91

50 most cited of 703 All 703 (JSON)