# WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Models

arXiv: https://arxiv.org/abs/2306.15087 · HTML: https://mlsanity.com/paper/2306.15087/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 31 references that are in the map.
- First posted on arXiv 26 Jun 2023.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- ["I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Dataset](https://mlsanity.com/paper/2205.09209/) (2022)
- [Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics](https://mlsanity.com/paper/2106.14574/) (2021)
- [Theories of "Gender" in NLP Bias Research](https://mlsanity.com/paper/2205.02526/) (2022)
- [Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities](https://mlsanity.com/paper/2102.04257/) (2021)
- [Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models](https://mlsanity.com/paper/2206.11684/) (2022)
- [Towards WinoQueer: Developing a Benchmark for Anti-Queer Bias in Large Language Models](https://mlsanity.com/paper/2206.11484/) (2022)

## Built on by (most cited first, 30 of 42)

- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directions](https://mlsanity.com/paper/2404.03264/) (2024)
- [Fairness in Large Language Models: A Taxonomic Survey](https://mlsanity.com/paper/2404.01349/) (2024)
- [Bias Testing and Mitigation in LLM-based Code Generation](https://mlsanity.com/paper/2309.14345/) (2023)
- [Navigating LLM Ethics: Advancements, Challenges, and Future Directions](https://mlsanity.com/paper/2406.18841/) (2024)
- [Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art](https://mlsanity.com/paper/2406.03930/) (2024)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [Robust Pronoun Fidelity with English LLMs: Are they Reasoning, Repeating, or Just Biased?](https://mlsanity.com/paper/2404.03134/) (2024)
- [A Piece of Theatre: Investigating How Teachers Design LLM Chatbots to Assist Adolescent Cyberbullying Education](https://mlsanity.com/paper/2402.17456/) (2024)
- [Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms](https://mlsanity.com/paper/2307.10223/) (2023)
- [BenchmarkCards: Standardized Documentation for Large Language Model Benchmarks](https://mlsanity.com/paper/2410.12974/) (2024)
- [GenderCARE: A Comprehensive Framework for Assessing and Reducing Gender Bias in Large Language Models](https://mlsanity.com/paper/2408.12494/) (2024)
- [HEARTS: A Holistic Framework for Explainable, Sustainable and Robust Text Stereotype Detection](https://mlsanity.com/paper/2409.11579/) (2024)
- [Laissez-Faire Harms: Algorithmic Biases in Generative Language Models](https://mlsanity.com/paper/2404.07475/) (2024)
- [The Root Shapes the Fruit: On the Persistence of Gender-Exclusive Harms in Aligned Language Models](https://mlsanity.com/paper/2411.03700/) (2024)
- [LangFair: A Python Package for Assessing Bias and Fairness in Large Language Model Use Cases](https://mlsanity.com/paper/2501.03112/) (2025)
- [FairTranslate: An English-French Dataset for Gender Bias Evaluation in Machine Translation by Overcoming Gender Binarity](https://mlsanity.com/paper/2504.15941/) (2025)
- [A Group Fairness Lens for Large Language Models](https://mlsanity.com/paper/2312.15478/) (2023)
- [Learning from Red Teaming: Gender Bias Provocation and Mitigation in Large Language Models](https://mlsanity.com/paper/2310.11079/) (2023)
- [BEADs: Bias Evaluation Across Domains](https://mlsanity.com/paper/2406.04220/) (2024)
- [Identity-related Speech Suppression in Generative AI Content Moderation](https://mlsanity.com/paper/2409.13725/) (2024)
- [Investigating Intersectional Bias in Large Language Models using Confidence Disparities in Coreference Resolution](https://mlsanity.com/paper/2508.07111/) (2025)
- [When Stereotypes GTG: The Impact of Predictive Text Suggestions on Gender Bias in Human-AI Co-Writing](https://mlsanity.com/paper/2409.20390/) (2024)
- [Data Defenses Against Large Language Models](https://mlsanity.com/paper/2410.13138/) (2024)
- [CALM: Culturally Self-Aware Language Models](https://mlsanity.com/paper/2601.03483/) (2026)
- [An Empirical Design Justice Approach to Identifying Ethical Considerations in the Intersection of Large Language Models and Social Robotics](https://mlsanity.com/paper/2406.06400/) (2024)
- [Exploring LGBTQ+ Bias in Generative AI Answers across Different Country and Religious Contexts](https://mlsanity.com/paper/2407.03473/) (2024)
- [Speciesism in Natural Language Processing Research](https://mlsanity.com/paper/2410.14194/) (2024)
- [Boardwalk Empire: How Generative AI is Revolutionizing Economic Paradigms](https://mlsanity.com/paper/2410.15212/) (2024)
- [An Empirically-grounded tool for Automatic Prompt Linting and Repair: A Case Study on Bias, Vulnerability, and Optimization in Developer Prompts](https://mlsanity.com/paper/2501.12521/) (2025)

## Abstract (arXiv)

We present WinoQueer: a benchmark specifically designed to measure whether large language models (LLMs) encode biases that are harmful to the LGBTQ+ community. The benchmark is community-sourced, via application of a novel method that generates a bias benchmark from a community survey. We apply our benchmark to several popular LLMs and find that off-the-shelf models generally do exhibit considerable anti-queer bias. Finally, we show that LLM bias against a marginalized community can be somewhat mitigated by finetuning on data written about or by members of that community, and that social media text written by community members is more effective than news text written about the community by non-members. Our method for community-in-the-loop benchmark development provides a blueprint for future researchers to develop community-driven, harms-grounded LLM benchmarks for other marginalized communities. Note: This version corrects a bug found in evaluation code after publication. General findings have not changed, but tables 5 and 6 and figure 1 have been corrected.