Learning from Red Teaming: Gender Bias Provocation and Mitigation in Large Language Models
On the map: LLM safety and security → Safety alignment, refusal and guardrails
Abstract (arXiv)
Recently, researchers have made considerable improvements in dialogue systems with the progress of large language models (LLMs) such as ChatGPT and GPT-4. These LLM-based chatbots encode the potential biases while retaining disparities that can harm humans during interactions. The traditional biases investigation methods often rely on human-written test cases. However, these test cases are usually expensive and limited. In this work, we propose a first-of-its-kind method that automatically generates test cases to detect LLMs' potential gender bias. We apply our method to three well-known LLMs and find that the generated test cases effectively identify the presence of biases. To address the biases identified, we propose a mitigation strategy that uses the generated test cases as demonstrations for in-context learning to circumvent the need for parameter fine-tuning. The experimental results show that LLMs generate fairer responses with the proposed approach.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 39 of its 51 references that are in the map. First posted on arXiv 17 Oct 2023.
Builds on
39 of 51 references.
References · 39
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
- 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
- 2022Red Teaming Language Models with Language Modelscited by 983
- 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
- 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
- 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
27 more
- 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
- 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
- 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
- 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
- 2019On Measuring Social Biases in Sentence Encoderscited by 397
- 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
- 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 299
- 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
- 2019Measuring Bias in Contextualized Word Representationscited by 275
- 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 230
- 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
- 2018Gender Bias in Neural Natural Language Processingcited by 210
- 2021Societal Biases in Language Generation: Progress and Challengescited by 201
- 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
- 2018Learning Gender-Neutral Word Embeddingscited by 169
- 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152
- 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
- 2020Towards Controllable Biases in Language Generationcited by 112
- 2019It's All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitutioncited by 102
- 2019Does Gender Matter? Towards Fairness in Dialogue Systemscited by 89
- 2021Language Model Evaluation Beyond Perplexitycited by 77
- 2020Investigating African-American Vernacular English in Transformer-Based Text Generationcited by 52
- 2022Perturbation Augmentation for Fairer NLPcited by 51
- 2020Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learningcited by 44
- 2023WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Modelscited by 42
- 2023Language Models Get a Gender Makeover: Mitigating Gender Bias with Few-Shot Data Interventionscited by 28
- 2022How sensitive are translation systems to extra contexts? Mitigating gender bias in Neural Machine Translation models through relevant contextscited by 10
Built on by
Citing papers per year: 2024: 2.
2 papers.