mlsanityData to 7 Oct 2026

Adversarial Scrubbing of Demographic Information for Text Classification

arXiv 2109.08613 ↗First posted 17 Sep 2021cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

Contextual representations learned by language models can often encode undesirable attributes, like demographic associations of the users, while being trained for an unrelated target task. We aim to scrub such undesirable attributes and learn fair representations while maintaining performance on the target task. In this paper, we present an adversarial learning framework "Adversarial Scrubber" (ADS), to debias contextual representations. We perform theoretical analysis to show that our framework converges without leaking demographic information under certain conditions. We extend previous evaluation techniques by evaluating debiasing performance using Minimum Description Length (MDL) probing. Experimental evaluations on 8 datasets show that ADS generates representations with minimal information about demographic attributes while being maximally informative about the target task.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 24 of its 44 references that are in the map. First posted on arXiv 17 Sep 2021.

Builds on

24 of 44 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  4. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  5. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  6. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  7. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  8. 2019Designing and Interpreting Probes with Control Taskscited by 608
  9. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  10. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  11. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  12. 2019Evaluating Gender Bias in Machine Translationcited by 247
12 more
  1. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  2. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 230
  3. 2016Demographic Dialectal Variation in Social Media: A Case Study of African-American Englishcited by 194
  4. 2018Adversarial Removal of Demographic Attributes from Text Datacited by 166
  5. 2018Reducing Gender Bias in Abusive Language Detectioncited by 152
  6. 2019Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overviewcited by 137
  7. 2020Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problemcited by 86
  8. 2020Multi-Dimensional Gender Bias Classificationcited by 80
  9. 2018Towards Robust and Privacy-preserving Text Representationscited by 75
  10. 2019Equalizing Gender Biases in Neural Machine Translation with Word Embeddings Techniquescited by 65
  11. 2020Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learningcited by 44
  12. 2021The Authors Matter: Understanding and Mitigating Implicit Bias in Deep Text Classificationcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 7; 2024: 1; 2025: 2; 2026: 1.

14 papers.

Sort:

  1. 2024Exploring Safety-Utility Trade-Offs in Personalized Language Modelscited by 26
  2. 2022Learning Fair Representations via Rate-Distortion Maximizationcited by 15
  3. 2023Robust Concept Erasure via Kernelized Rate-Distortion Maximizationcited by 10
  4. 2022Fair NLP Models with Differentially Private Text Encoderscited by 9
  5. 2023Gradient-Free Privacy Leakage in Federated Language Models through Selective Weight Tamperingcited by 9
  6. 2022Sustaining Fairness via Incremental Learningcited by 5
  7. 2023Uncovering and Categorizing Social Biases in Text-to-SQLcited by 5
  8. 2025Fundamental Limits of Perfect Concept Erasurecited by 5
  9. 2023Enhancing Group Fairness in Online Settings Using Oblique Decision Forestscited by 3
  10. 2023Dual-Teacher De-biasing Distillation Framework for Multi-domain Fake News Detectioncited by 2
  11. 2023A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classificationcited by 1
  12. 2026Obliviator Reveals the Cost of Nonlinear Guardedness in Concept Erasurecited by 1
2 more
  1. 2023Transferring Fairness using Multi-Task Learning with Limited Demographic Informationcited by 0
  2. 2025Adversarially Probing Cross-Family Sound Symbolism in 27 Languagescited by 0