mlsanityData to 7 Oct 2026

Co2PT: Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuning

arXiv 2310.12490 ↗First posted 19 Oct 2023cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

Pre-trained Language Models are widely used in many important real-world applications. However, recent studies show that these models can encode social biases from large pre-training corpora and even amplify biases in downstream applications. To address this challenge, we propose Co2PT, an efficient and effective debias-while-prompt tuning method for mitigating biases via counterfactual contrastive prompt tuning on downstream tasks. Our experiments conducted on three extrinsic bias benchmarks demonstrate the effectiveness of Co2PT on bias mitigation during the prompt tuning process and its adaptability to existing upstream debiased language models. These findings indicate the strength of Co2PT and provide promising avenues for further enhancement in bias mitigation on downstream tasks.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 34 of its 43 references that are in the map. First posted on arXiv 19 Oct 2023.

Builds on

34 of 43 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  4. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  5. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  6. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  7. 2021GPT Understands, Toocited by 715
  8. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  9. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  10. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  11. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  12. 2018Gender Bias in Coreference Resolutioncited by 438
22 more
  1. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  2. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  3. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  4. 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 222
  5. 2018Gender Bias in Neural Natural Language Processingcited by 210
  6. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  7. 2020Towards Debiasing Sentence Representationscited by 170
  8. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  9. 2019On Measuring and Mitigating Biased Inferences of Word Embeddingscited by 112
  10. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
  11. 2021Sustainable Modular Debiasing of Language Modelscited by 98
  12. 2021FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoderscited by 75
  13. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  14. 2019What's in a Name? Reducing Bias in Bios without Access to Protected Attributescited by 51
  15. 2021Diverse Adversaries for Mitigating Bias in Trainingcited by 48
  16. 2022Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Taskscited by 48
  17. 2018Feature-Wise Bias Amplificationcited by 31
  18. 2022MABEL: Attenuating Gender Bias using Textual Entailment Datacited by 31
  19. 2023Prompt Tuning Pushes Farther, Contrastive Learning Pulls Closer: A Two-Stage Approach to Mitigate Social Biasescited by 21
  20. 2022Closed-book Question Generation via Contrastive Learningcited by 6
  21. 2023PromptAttack: Probing Dialogue State Trackers with Adversarial Promptscited by 6
  22. 2022Continuous Prompt Tuning Based Textual Entailment Model for E-commerce Entity Typingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 5; 2025: 7; 2026: 1.

15 papers.

Sort:

By area:

  1. 2023The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensivenesscited by 63
  2. 2025FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steeringcited by 37
  3. 2023Probing Explicit and Implicit Gender Bias through LLM Conditional Text Generationcited by 31
  4. 2024Navigating LLM Ethics: Advancements, Challenges, and Future Directionscited by 28
  5. 2024Large Language Model Bias Mitigation from the Perspective of Knowledge Editingcited by 16
  6. 2024Identifying and Mitigating Social Bias Knowledge in Language Modelscited by 13
  7. 2024A Contrastive Learning Approach to Mitigate Bias in Speech Modelscited by 9
  8. 2024Mitigating Biases of Large Language Models in Stance Detection with Counterfactual Augmented Calibrationcited by 5
  9. 2025C2PO: Diagnosing and Disentangling Bias Shortcuts in LLMscited by 5
  10. 2025Investigating Hallucination in Conversations for Low Resource Languagescited by 2
  11. 2025AMBEDKAR-A Multi-level Bias Elimination through a Decoding Approach with Knowledge Augmentation for Robust Constitutional Alignment of Language Modelscited by 2
  12. 2025Masculine Defaults via Gendered Discourse in Podcasts and Large Language Modelscited by 0
3 more
  1. 2025Debiasing Large Language Models in Thai Political Stance Detection via Counterfactual Calibrationcited by 0
  2. 2025Improving Fairness of Large Language Model-Based ICU Mortality Prediction via Case-Based Promptingcited by 0
  3. 2026Faithful-Patchscopes: Understanding and Mitigating Model Bias in Hidden Representations Explanation of Large Language Modelscited by 0