mlsanityData to 14 Oct 2026

Local Contrastive Editing of Gender Stereotypes

arXiv 2410.17739 ↗First posted 23 Oct 2024cs.CL cs.CYcited by 3

Not on the map. Why?

Abstract (arXiv)

Stereotypical bias encoded in language models (LMs) poses a threat to safe language technology, yet our understanding of how bias manifests in the parameters of LMs remains incomplete. We introduce local contrastive editing that enables the localization and editing of a subset of weights in a target model in relation to a reference model. We deploy this approach to identify and modify subsets of weights that are associated with gender stereotypes in LMs. Through a series of experiments, we demonstrate that local contrastive editing can precisely localize and control a small subset (< 0.5%) of weights that encode gender bias. Our work (i) advances our understanding of how stereotypical biases can manifest in the parameter space of LMs and (ii) opens up new avenues for developing parameter-efficient strategies for controlling model properties in a contrastive manner.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 30 of its 44 references that are in the map. First posted on arXiv 23 Oct 2024.

Builds on

30 of 44 references.

References · 30

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  3. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  4. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  5. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  6. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  7. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  8. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  9. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  10. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 659
  11. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  12. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
18 more
  1. 2020The Lottery Ticket Hypothesis for Pre-trained BERT Networkscited by 275
  2. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  3. 2022Patching open-vocabulary models by interpolating weightscited by 190
  4. 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
  5. 2020When BERT Plays the Lottery, All Tickets Are Winningcited by 114
  6. 2021Sustainable Modular Debiasing of Language Modelscited by 100
  7. 2023Knowledge is a Region in Weight Space for Fine-tuned Language Modelscited by 58
  8. 2024Disclosure and Mitigation of Gender Bias in LLMscited by 49
  9. 2020Multi-SimLex: A Large-Scale Evaluation of Multilingual and Cross-Lingual Lexical Semantic Similaritycited by 38
  10. 2019A General Framework for Implicit and Explicit Debiasing of Distributional Word Vector Spacescited by 36
  11. 2023Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Modelcited by 22
  12. 2022Gender Biases and Where to Find Them: Exploring Gender Bias in Pre-Trained Transformer-based Language Models Using Movement Pruningcited by 21
  13. 2022Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworkscited by 18
  14. 2021Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you?cited by 16
  15. 2022Debiasing Masks: A New Framework for Shortcut Mitigation in NLUcited by 16
  16. 2023Examining Modularity in Multilingual LMs via Language-Specialized Subnetworkscited by 13
  17. 2023In-Depth Look at Word Filling Societal Bias Measurescited by 8
  18. 2020Mitigating Gender Bias in Machine Learning Data Setscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2026Neuron-Level Interventions for Gendered and Gender-Neutral Generation in Language Modelscited by 2
  2. 2025Gender Encoding Patterns in Pretrained Language Model Representationscited by 1
  3. 2025An Empirical Survey of Model Merging Algorithms for Social Bias Mitigationcited by 0