mlsanityData to 14 Oct 2026

Towards Effective Discrimination Testing for Generative AI

arXiv 2412.21052 ↗First posted 30 Dec 2024cs.LG cs.AI cs.CYcited by 5

Not on the map. Why?

Abstract (arXiv)

Generative AI (GenAI) models present new challenges in regulating against discriminatory behavior. In this paper, we argue that GenAI fairness research still has not met these challenges; instead, a significant gap remains between existing bias assessment methods and regulatory goals. This leads to ineffective regulation that can allow deployment of reportedly fair, yet actually discriminatory, GenAI systems. Towards remedying this problem, we connect the legal and technical literature around GenAI bias evaluation and identify areas of misalignment. Through four case studies, we demonstrate how this misalignment between fairness testing techniques and regulatory goals can result in discriminatory outcomes in real-world deployments, especially in adaptive or complex environments. We offer practical recommendations for improving discrimination testing to better align with regulatory goals and enhance the reliability of fairness assessments in future deployments.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 54 of its 102 references that are in the map. First posted on arXiv 30 Dec 2024.

Builds on

54 of 102 references.

References · 54

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  7. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  8. 2020Extracting Training Data from Large Language Modelscited by 2,189
  9. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  10. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  11. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  12. 2022Red Teaming Language Models with Language Modelscited by 1,026
38 more
  1. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  2. 2023Extracting Training Data from Diffusion Modelscited by 732
  3. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  4. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 499
  5. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 449
  6. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 349
  7. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  8. 2019Evaluating Gender Bias in Machine Translationcited by 249
  9. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 231
  10. 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
  11. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 226
  12. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 214
  13. 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Promptscited by 205
  14. 2021What Will it Take to Fix Benchmarking in Natural Language Understanding?cited by 203
  15. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 199
  16. 2024Foundational Challenges in Assuring Alignment and Safety of Large Language Modelscited by 199
  17. 2023Gender bias and stereotypes in Large Language Modelscited by 187
  18. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 175
  19. 2020Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claimscited by 165
  20. 2023Benchmarking Cognitive Biases in Large Language Models as Evaluatorscited by 150
  21. 2019A Question-Entailment Approach to Question Answeringcited by 134
  22. 2018Assessing Gender Bias in Machine Translation -- A Case Study with Google Translatecited by 121
  23. 2023FLIRT: Feedback Loop In-context Red Teamingcited by 83
  24. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 83
  25. 2024Latent Guard: a Safety Framework for Text-to-image Generationcited by 71
  26. 2023Style Over Substance: Evaluation Biases for Large Language Modelscited by 70
  27. 2024PersonalLLM: Tailoring LLMs to Individual Preferencescited by 60
  28. 2024What's in a Name? Auditing Large Language Models for Race and Gender Biascited by 51
  29. 2024Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrievalcited by 51
  30. 2024Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluationcited by 41
  31. 2021Leave-one-out Unfairnesscited by 36
  32. 2023Evaluation of African American Language Bias in Natural Language Generationcited by 36
  33. 2024ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Userscited by 35
  34. 2023ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languagescited by 34
  35. 2022Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimizationcited by 33
  36. 2023On The Impact of Machine Learning Randomness on Group Fairnesscited by 26
  37. 2024Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generationcited by 20
  38. 2024Auditing the Use of Language Models to Guide Hiring Decisionscited by 20

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025Guiding LLM Decision-Making with Fairness Reward Modelscited by 9
  2. 2025Breaking Down Bias: On The Limits of Generalizable Pruning Strategiescited by 2
  3. 2026Resume-ing Control: (Mis)Perceptions of Agency Around GenAI Use in Recruiting Workflowscited by 1
  4. 2025It's complicated. The relationship of algorithmic fairness and non-discrimination provisions for high-risk systems in the EU AI Actcited by 0
  5. 2025Test-Time Warmup for Multimodal Large Language Modelscited by 0