mlsanityData to 7 Oct 2026

SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models

arXiv 2312.07492 ↗First posted 12 Dec 2023cs.CL cs.AI cs.CYcited by 14

Not on the map. Why?

Abstract (arXiv)

Current datasets for unwanted social bias auditing are limited to studying protected demographic features such as race and gender. In this work, we introduce a comprehensive benchmark that is meant to capture the amplification of social bias, via stigmas, in generative language models. Taking inspiration from social science research, we start with a documented list of 93 US-centric stigmas and curate a question-answering (QA) dataset which involves simple social situations. Our benchmark, SocialStigmaQA, contains roughly 10K prompts, with a variety of prompt styles, carefully constructed to systematically test for both social bias and model robustness. We present results for SocialStigmaQA with two open source generative language models and we find that the proportion of socially biased output ranges from 45% to 59% across a variety of decoding strategies and prompting styles. We demonstrate that the deliberate design of the templates in our benchmark (e.g., adding biasing text to the prompt or using different verbs that change the answer that indicates bias) impacts the model tendencies to generate socially biased output. Additionally, through manual evaluation, we discover problematic patterns in the generated chain-of-thought output that range from subtle bias to lack of reasoning. Warning: This paper contains examples of text which are toxic, biased, and potentially harmful.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 17 of its 38 references that are in the map. First posted on arXiv 12 Dec 2023.

Builds on

17 of 38 references.

References · 17

  1. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  2. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  3. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  4. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  5. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  6. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  7. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  8. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  9. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  10. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 340
  11. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  12. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 253
5 more
  1. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  2. 2023Bias Against 93 Stigmatized Groups in Masked Language Models and Downstream Sentiment Classification Taskscited by 51
  3. 2021Your fairness may vary: Pretrained language model fairness in toxic text classificationcited by 38
  4. 2021The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Failcited by 38
  5. 2022The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarkscited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 4; 2026: 3.

14 papers.

Sort:

By area:

  1. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  2. 2024Building Guardrails for Large Language Modelscited by 61
  3. 2025Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agentscited by 26
  4. 2024SeeGULL Multilingual: a Dataset of Geo-Culturally Situated Stereotypescited by 24
  5. 2024Language Models in Dialogue: Conversational Maxims for Human-AI Interactionscited by 21
  6. 2024Multi-Level Explanations for Generative Language Modelscited by 11
  7. 2024Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitationscited by 10
  8. 2026Uncovering Political Bias in Large Language Models using Parliamentary Voting Recordscited by 7
  9. 2025Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directionscited by 5
  10. 2025Attention Pruning: Automated Fairness Repair of Language Models via Surrogate Simulated Annealingcited by 4
  11. 2026Investigating Social Bias Changes in Quantized Language Modelscited by 4
  12. 2024When in Doubt, Cascade: Towards Building Efficient and Capable Guardrailscited by 1
2 more
  1. 2025Identifying Features Associated with Bias Against 93 Stigmatized Groups in Language Models and Guardrail Model Safety Mitigationcited by 1
  2. 2026The Safeguard Worked. Is the LLM System Safer?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ImplicitBBQ: Benchmarking Implicit Bias in Large Language Models through Characteristic Based Cues
  2. 2026Evaluating Social Bias in RAG Systems: When External Context Helps and Reasoning Hurts
  3. 2024BiasAlert: A Plug-and-play Tool for Social Bias Detection in LLMs
  4. 2025Mitigating Social Bias in English and Urdu Language Models Using PRM-Guided Candidate Selection and Sequential Refinement
  5. 2024Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals
  6. 2025Investigating Thinking Behaviours of Reasoning-Based Language Models for Social Bias Mitigation
  7. 2025Evaluating the Effect of Retrieval Augmentation on Social Biases
  8. 2024Uncovering Bias in Large Vision-Language Models with Counterfactuals