mlsanityData to 14 Oct 2026

Reducing Sentiment Bias in Language Models via Counterfactual Evaluation

arXiv 1911.03064 ↗First posted 8 Nov 2019cs.CL cs.CY cs.LGcited by 148

Not on the map. Why?

Abstract (arXiv)

Advances in language modeling architectures and the availability of large text corpora have driven progress in automatic text generation. While this results in models capable of generating coherent texts, it also prompts models to internalize social biases present in the training corpus. This paper aims to quantify and reduce a particular type of bias exhibited by language models: bias in the sentiment of generated text. Given a conditioning context (e.g., a writing prompt) and a language model, we analyze if (and how) the sentiment of the generated text is affected by changes in values of sensitive attributes (e.g., country names, occupations, genders) in the conditioning context using a form of counterfactual evaluation. We quantify sentiment bias by adopting individual and group fairness metrics from the fair machine learning literature, and demonstrate that large-scale models trained on two different corpora (news articles, and Wikipedia) exhibit considerable levels of bias. We then propose embedding and sentiment prediction-derived regularization on the language model's latent representations. The regularizations improve fairness metrics while retaining comparable levels of perplexity and semantic similarity.

Key facts. Cited by 148 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 19; 2022: 31; 2023: 34; 2024: 37; 2025: 23; 2026: 4.

148 papers.

Sort:

By area:

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  3. 2022Red Teaming Language Models with Language Modelscited by 1,026
  4. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  5. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  6. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 296
  7. 2021Societal Biases in Language Generation: Progress and Challengescited by 202
  8. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 178
  9. 2023The Capacity for Moral Self-Correction in Large Language Modelscited by 173
  10. 2021Challenges in Detoxifying Language Modelscited by 169
  11. 2021Alignment of Language Agentscited by 168
  12. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 162
38 more
  1. 2023Debiasing Vision-Language Models via Biased Promptscited by 149
  2. 2021Self-Supervised Representation Learning: Introduction, Advances and Challengescited by 148
  3. 2023Co-Writing with Opinionated Language Models Affects Users' Viewscited by 148
  4. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 129
  5. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 112
  6. 2024Cognitive Bias in Decision-Making with LLMscited by 106
  7. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 98
  8. 2023A Survey on Fairness in Large Language Modelscited by 95
  9. 2021Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metricscited by 83
  10. 2021On Measures of Biases and Harms in NLPcited by 81
  11. 2021Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Casescited by 77
  12. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  13. 2021Mitigating Political Bias in Language Models Through Reinforced Calibrationcited by 73
  14. 2024Red Teaming Visual Language Modelscited by 65
  15. 2022Theories of "Gender" in NLP Bias Researchcited by 62
  16. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 58
  17. 2022Perturbation Augmentation for Fairer NLPcited by 52
  18. 2023Bias Against 93 Stigmatized Groups in Masked Language Models and Downstream Sentiment Classification Taskscited by 51
  19. 2022Evaluation Gaps in Machine Learning Practicecited by 49
  20. 2022Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Modelscited by 49
  21. 2024Disclosure and Mitigation of Gender Bias in LLMscited by 49
  22. 2022A Causal Lens for Controllable Text Generationcited by 45
  23. 2022Identifying and Manipulating the Personality Traits of Language Modelscited by 40
  24. 2022Quantifying Social Biases Using Templates is Unreliablecited by 39
  25. 2023Bias of AI-Generated Content: An Examination of News Produced by Large Language Modelscited by 38
  26. 2023Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systemscited by 37
  27. 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
  28. 2022Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimizationcited by 33
  29. 2023Probing Explicit and Implicit Gender Bias through LLM Conditional Text Generationcited by 31
  30. 2024Recent Advances in Generative AI and Large Language Models: Current Status, Challenges, and Perspectivescited by 30
  31. 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 28
  32. 2024Identifying Implicit Social Biases in Vision-Language Modelscited by 27
  33. 2021Intersectional Bias in Causal Language Modelscited by 24
  34. 2021Detect and Perturb: Neutral Rewriting of Biased and Sensitive Text via Gradient-based Decodingcited by 23
  35. 2022Unit Testing for Concepts in Neural Networkscited by 23
  36. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  37. 2022LMentry: A Language Model Benchmark of Elementary Language Taskscited by 22
  38. 2023Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Taskscited by 22

50 most cited of 148 All 148 (JSON)