mlsanityData to 7 Oct 2026

BiasTestGPT: Using ChatGPT for Social Bias Testing of Language Models

arXiv 2302.07371 ↗First posted 14 Feb 2023cs.CL cs.CYcited by 8

Not on the map. Why?

Abstract (arXiv)

Pretrained Language Models (PLMs) harbor inherent social biases that can result in harmful real-world implications. Such social biases are measured through the probability values that PLMs output for different social groups and attributes appearing in a set of test sentences. However, bias testing is currently cumbersome since the test sentences are generated either from a limited set of manual templates or need expensive crowd-sourcing. We instead propose using ChatGPT for the controllable generation of test sentences, given any arbitrary user-specified combination of social groups and attributes appearing in the test sentences. When compared to template-based methods, our approach using ChatGPT for test sentence generation is superior in detecting social bias, especially in challenging settings such as intersectional biases. We present an open-source comprehensive bias testing framework (BiasTestGPT), hosted on HuggingFace, that can be plugged into any open-source PLM for bias testing. User testing with domain experts from various fields has shown their interest in being able to test modern AI for social biases. Our tool has significantly improved their awareness of such biases in PLMs, proving to be learnable and user-friendly. We thus enable seamless open-ended social bias testing of PLMs by domain experts through an automatic large-scale generation of diverse test sentences for any combination of social categories and attributes.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 47 of its 123 references that are in the map. First posted on arXiv 14 Feb 2023.

Builds on

47 of 123 references.

References · 47

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  4. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  5. 2018Model Cards for Model Reportingcited by 1,068
  6. 2019Publicly Available Clinical BERT Embeddingscited by 995
  7. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  8. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  9. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  10. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  11. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  12. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
35 more
  1. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  2. 2018Gender Bias in Coreference Resolutioncited by 438
  3. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  4. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  5. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  6. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  7. 2019A Multiscale Visualization of Attention in the Transformer Modelcited by 292
  8. 2019Measuring Bias in Contextualized Word Representationscited by 275
  9. 2019The What-If Tool: Interactive Probing of Machine Learning Modelscited by 207
  10. 2023The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientationcited by 183
  11. 2019Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasetscited by 173
  12. 2020Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biasescited by 167
  13. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  14. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
  15. 2019On Measuring and Mitigating Biased Inferences of Word Embeddingscited by 112
  16. 2019FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learningcited by 93
  17. 2022Evaluating Human-Language Model Interactioncited by 90
  18. 2020Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Biascited by 82
  19. 2020Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddingscited by 79
  20. 2022Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Modelscited by 67
  21. 2022Interactive Model Cards: A Human-Centered Approach to Model Documentationcited by 61
  22. 2022Handling Bias in Toxic Speech Detection: A Surveycited by 60
  23. 2020PowerTransformer: Unsupervised Controllable Revision for Biased Language Correctioncited by 59
  24. 2022Quantifying Social Biases Using Templates is Unreliablecited by 38
  25. 2019Natural Language Processing of Clinical Notes on Chronic Diseases: Systematic Reviewcited by 36
  26. 2022The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarkscited by 31
  27. 2021Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biasescited by 21
  28. 2023ChatGPT and Other Large Language Models as Evolutionary Engines for Online Interactive Collaborative Game Designcited by 20
  29. 2022ConceptExplainer: Interactive Explanation for Deep Neural Networks from a Concept Perspectivecited by 18
  30. 2022Gendered Mental Health Stigma in Masked Language Modelscited by 17
  31. 2021T3-Vis: a visual analytic framework for Training and fine-Tuning Transformers in NLPcited by 11
  32. 2022Can You Label Less by Using Out-of-Domain Data? Active & Transfer Learning with Few-shot Instructionscited by 7
  33. 2022RMExplorer: A Visual Analytics Approach to Explore the Performance and the Fairness of Disease Risk Models on Population Subgroupscited by 6
  34. 2021DebIE: A Platform for Implicit and Explicit Debiasing of Word Embedding Spacescited by 5
  35. 2021Assessing gender bias in medical and scientific masked language models with StereoSetcited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 5.

8 papers.

  1. 2023Denevil: Towards Deciphering and Navigating the Ethical Values of Large Language Models via Instruction Learningcited by 26
  2. 2024Investigating Implicit Bias in Large Language Models: A Large-Scale Study of Over 50 LLMscited by 25
  3. 2024Walking in Others' Shoes: How Perspective-Taking Guides Large Language Models in Reducing Toxicity and Biascited by 17
  4. 2025AdAEM: An Adaptively and Automated Extensible Measurement of LLMs' Value Differencecited by 4
  5. 2025Self-Anchored Attention Model for Sample-Efficient Classification of Prosocial Text Chatcited by 1
  6. 2025Exploring Gender Bias Beyond Occupational Titlescited by 0
  7. 2025BTC-SAM: Leveraging LLMs for Generation of Bias Test Cases for Sentiment Analysis Modelscited by 0
  8. 2025Bias Testing and Mitigation in Black Box LLMs using Metamorphic Relationscited by 0