mlsanityData to 14 Oct 2026

Towards Debiasing Sentence Representations

arXiv 2007.08100 ↗First posted 16 Jul 2020cs.CL cs.LGcited by 171

Not on the map. Why?

Abstract (arXiv)

As natural language processing methods are increasingly deployed in real-world scenarios such as healthcare, legal systems, and social science, it becomes necessary to recognize the role they potentially play in shaping social biases and stereotypes. Previous work has revealed the presence of social biases in widely used word embeddings involving gender, race, religion, and other social constructs. While some methods were proposed to debias these word-level embeddings, there is a need to perform debiasing at the sentence-level given the recent shift towards new contextualized sentence representations such as ELMo and BERT. In this paper, we investigate the presence of social biases in sentence-level representations and propose a new method, Sent-Debias, to reduce these biases. We show that Sent-Debias is effective in removing biases, and at the same time, preserves performance on sentence-level downstream tasks such as sentiment analysis, linguistic acceptability, and natural language understanding. We hope that our work will inspire future research on characterizing and removing social biases from widely adopted sentence representations for fairer NLP.

Key facts. Cited by 171 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 22; 2022: 26; 2023: 43; 2024: 31; 2025: 31; 2026: 18.

171 papers.

Sort:

By area:

  1. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  2. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  3. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 361
  4. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 296
  5. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 181
  6. 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
  7. 2023Debiasing Vision-Language Models via Biased Promptscited by 149
  8. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 146
  9. 2021Learning Debiased Representation via Disentangled Feature Augmentationcited by 116
  10. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 97
  11. 2023A Survey on Fairness in Large Language Modelscited by 95
  12. 2021Mitigating Language-Dependent Ethnic Bias in BERTcited by 87
38 more
  1. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 84
  2. 2021Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metricscited by 83
  3. 2024Bias in Large Language Models: Origin, Evaluation, and Mitigationcited by 77
  4. 2021FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoderscited by 75
  5. 2023Fast Model Debias with Machine Unlearningcited by 72
  6. 2023Word Embeddings Are Steers for Language Modelscited by 70
  7. 2022Theories of "Gender" in NLP Bias Researchcited by 62
  8. 2022Handling Bias in Toxic Speech Detection: A Surveycited by 61
  9. 2021Directional Bias Amplificationcited by 54
  10. 2024Disclosure and Mitigation of Gender Bias in LLMscited by 49
  11. 2022Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Taskscited by 48
  12. 2024Thinking Fair and Slow: On the Efficacy of Structured Prompts for Debiasing Language Modelscited by 45
  13. 2025FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steeringcited by 40
  14. 2023AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Riskscited by 35
  15. 2021StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfercited by 34
  16. 2022Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimizationcited by 33
  17. 2022Learning Fair Representation via Distributional Contrastive Disentanglementcited by 31
  18. 2022MABEL: Attenuating Gender Bias using Textual Entailment Datacited by 31
  19. 2023Probing Explicit and Implicit Gender Bias through LLM Conditional Text Generationcited by 31
  20. 2024GenderBias-VL: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probingcited by 31
  21. 2023OpinionGPT: Modelling Explicit Biases in Instruction-Tuned LLMscited by 29
  22. 2022Toward Fairness in Speech Recognition: Discovery and mitigation of performance disparitiescited by 28
  23. 2023Language Models Get a Gender Makeover: Mitigating Gender Bias with Few-Shot Data Interventionscited by 28
  24. 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 28
  25. 2022From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible MLcited by 27
  26. 2023Survey of Social Bias in Vision-Language Modelscited by 26
  27. 2021Conditional Contrastive Learning for Improving Fairness in Self-Supervised Learningcited by 25
  28. 2021RuleBert: Teaching Soft Rules to Pre-trained Language Modelscited by 25
  29. 2023Analyzing And Editing Inner Mechanisms Of Backdoored Language Modelscited by 23
  30. 2022SODAPOP: Open-Ended Discovery of Social Biases in Social Commonsense Reasoning Modelscited by 22
  31. 2022Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Surveycited by 22
  32. 2022ADEPT: A DEbiasing PrompT Frameworkcited by 21
  33. 2023Prompt Tuning Pushes Farther, Contrastive Learning Pulls Closer: A Two-Stage Approach to Mitigate Social Biasescited by 21
  34. 2021Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classificationcited by 20
  35. 2023Evaluating Biased Attitude Associations of Language Models in an Intersectional Contextcited by 20
  36. 2023Weakly Supervised Detection of Hallucinations in LLM Activationscited by 20
  37. 2024Fairness and Bias in Multimodal AI: A Surveycited by 20
  38. 2022Naturalistic Causal Probing for Morpho-Syntaxcited by 19

50 most cited of 171 All 171 (JSON)