mlsanityData to 7 Oct 2026

Detect and Perturb: Neutral Rewriting of Biased and Sensitive Text via Gradient-based Decoding

arXiv 2109.11708 ↗First posted 24 Sep 2021cs.CLcited by 23

Not on the map. Why?

Abstract (arXiv)

Written language carries explicit and implicit biases that can distract from meaningful signals. For example, letters of reference may describe male and female candidates differently, or their writing style may indirectly reveal demographic characteristics. At best, such biases distract from the meaningful content of the text; at worst they can lead to unfair outcomes. We investigate the challenge of re-generating input sentences to 'neutralize' sensitive attributes while maintaining the semantic meaning of the original text (e.g. is the candidate qualified?). We propose a gradient-based rewriting framework, Detect and Perturb to Neutralize (DEPEN), that first detects sensitive components and masks them for regeneration, then perturbs the generation model at decoding time under a neutralizing constraint that pushes the (predicted) distribution of sensitive attributes towards a uniform distribution. Our experiments in two different scenarios show that DEPEN can regenerate fluent alternatives that are neutral in the sensitive attribute while maintaining the semantics of other attributes.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 24 of its 36 references that are in the map. First posted on arXiv 24 Sep 2021.

Builds on

24 of 36 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  3. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  4. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  5. 2018Gender Bias in Neural Natural Language Processingcited by 210
  6. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  7. 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
  8. 2018Disentangled Representation Learning for Non-Parallel Text Style Transfercited by 127
  9. 2018Style Transfer Through Back-Translationcited by 124
  10. 2020Learning to Faithfully Rationalize by Constructioncited by 113
  11. 2020Towards Controllable Biases in Language Generationcited by 112
  12. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
12 more
  1. 2019Automatically Neutralizing Subjective Bias in Textcited by 105
  2. 2019Does Gender Matter? Towards Fairness in Dialogue Systemscited by 89
  3. 2020Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problemcited by 86
  4. 2019Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representationcited by 84
  5. 2019Transforming Delete, Retrieve, Generate Approach for Controlled Text Style Transfercited by 81
  6. 2020PowerTransformer: Unsupervised Controllable Revision for Biased Language Correctioncited by 59
  7. 2020Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learningcited by 44
  8. 2021Ask what's missing and what's useful: Improving Clarification Question Generation using Global Knowledgecited by 40
  9. 2019Pre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoderscited by 27
  10. 2018Adversarial Decomposition of Text Representationcited by 18
  11. 2021Unsupervised Enrichment of Persona-grounded Dialog with Background Storiescited by 14
  12. 2021Improving Sequence-to-Sequence Pre-training via Sequence Span Rewritingcited by 13

Built on by

Cited by year

Citing papers per year: 2022: 6; 2023: 6; 2024: 7; 2025: 4.

23 papers.

Sort:

By area:

  1. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  2. 2024Cognitive Bias in Decision-Making with LLMscited by 104
  3. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  4. 2022Text Style Transfer for Bias Mitigation using Masked Language Modelingcited by 27
  5. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  6. 2023MedEval: A Multi-Level, Multi-Task, and Multi-Domain Medical Benchmark for Language Model Evaluationcited by 19
  7. 2022Controlling Bias Exposure for Fair Interpretable Predictionscited by 18
  8. 2023Targeted Data Generation: Finding and Fixing Model Weaknessescited by 15
  9. 2023Tackling Bias in Pre-trained Language Models: Current Trends and Under-represented Societiescited by 11
  10. 2023The Media Bias Taxonomy: A Systematic Literature Review on the Forms and Automated Detection of Media Biascited by 11
  11. 2024Potential and Challenges of Model Editing for Social Debiasingcited by 10
  12. 2022Synthetic Pre-Training Tasks for Neural Machine Translationcited by 8
11 more
  1. 2022InterFair: Debiasing with Natural Language Feedback for Fair Interpretable Predictionscited by 7
  2. 2025Quantifying Cognitive Bias Induction in LLM-Generated Contentcited by 6
  3. 2024Hire Me or Not? Examining Language Model's Behavior with Occupation Attributescited by 5
  4. 2024Mitigating Gender Bias in Code Large Language Models via Model Editingcited by 3
  5. 2022Style Transfer as Data Augmentation: A Case Study on Named Entity Recognitioncited by 2
  6. 2023"Nothing Abnormal": Disambiguating Medical Reports via Contrastive Knowledge Infusioncited by 2
  7. 2025LFTF: Locating First and Then Fine-Tuning for Mitigating Gender Bias in Large Language Modelscited by 2
  8. 2024Analyzing Sentiment Polarity Reduction in News Presentation through Contextual Perturbation and Large Language Modelscited by 1
  9. 2024A Note on Bias to Completecited by 1
  10. 2025GeNRe: A French Gender-Neutral Rewriting System Using Collective Nounscited by 1
  11. 2025DeCAP: Context-Adaptive Prompt Generation for Debiasing Zero-shot Question Answering in Large Language Modelscited by 0