mlsanityData to 7 Oct 2026

Hey GPT, Can You be More Racist? Analysis from Crowdsourced Attempts to Elicit Biased Content from Generative AI

arXiv 2410.15467 ↗First posted 20 Oct 2024cs.CL cs.AI cs.HCcited by 6

Not on the map. Why?

Abstract (arXiv)

The widespread adoption of large language models (LLMs) and generative AI (GenAI) tools across diverse applications has amplified the importance of addressing societal biases inherent within these technologies. While the NLP community has extensively studied LLM bias, research investigating how non-expert users perceive and interact with biases from these systems remains limited. As these technologies become increasingly prevalent, understanding this question is crucial to inform model developers in their efforts to mitigate bias. To address this gap, this work presents the findings from a university-level competition, which challenged participants to design prompts for eliciting biased outputs from GenAI tools. We quantitatively and qualitatively analyze the competition submissions and identify a diverse set of biases in GenAI and strategies employed by participants to induce bias in GenAI. Our finding provides unique insights into how non-expert users perceive and interact with biases from GenAI tools.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 18 of its 41 references that are in the map. First posted on arXiv 20 Oct 2024.

Builds on

18 of 41 references.

References · 18

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  3. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  4. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  5. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 160
  6. 2018Aequitas: A Bias and Fairness Audit Toolkitcited by 145
  7. 2018The Frontiers of Fairness in Machine Learningcited by 139
  8. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 97
  9. 2021Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metricscited by 83
  10. 2021On Measures of Biases and Harms in NLPcited by 81
  11. 2024Disclosure and Mitigation of Gender Bias in LLMscited by 47
  12. 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 27
6 more
  1. 2024Competition Report: Finding Universal Jailbreak Backdoors in Aligned LLMscited by 27
  2. 2024Do Generative AI Models Output Harm while Representing Non-Western Cultures: Evidence from A Community-Centered Approachcited by 25
  3. 2024Dataset and Lessons Learned from the 2024 SaTML LLM Capture-the-Flag Competitioncited by 20
  4. 2024Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAIcited by 14
  5. 2023Towards a Holistic Approach: Understanding Sociodemographic Biases in NLP Models using an Interdisciplinary Lenscited by 9
  6. 2024Are Large Language Models Really Bias-Free? Jailbreak Prompts for Assessing Adversarial Robustness to Bias Elicitationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025Understanding and Mitigating Bias Inheritance in LLM-based Data Augmentation on Downstream Taskscited by 9
  2. 2025Attention Pruning: Automated Fairness Repair of Language Models via Surrogate Simulated Annealingcited by 4
  3. 2025On the Performance of LLMs for Real Estate Appraisalcited by 3
  4. 2025Can LLMs Rank the Harmfulness of Smaller LLMs? We are Not There Yetcited by 0
  5. 2026To Build or Not to Build? Factors that Lead to Non-Development or Abandonment of AI Systemscited by 0
  6. 2026Beyond Benchmarks: Exposing the Hidden Crisis in Bangla Hate Speech Detectioncited by 0