mlsanityData to 14 Oct 2026

Shielded Representations: Protecting Sensitive Attributes Through Iterative Gradient-Based Projection

arXiv 2305.10204 ↗First posted 17 May 2023cs.CL cs.AIcited by 25

Not on the map. Why?

Abstract (arXiv)

Natural language processing models tend to learn and encode social biases present in the data. One popular approach for addressing such biases is to eliminate encoded information from the model's representations. However, current methods are restricted to removing only linearly encoded information. In this work, we propose Iterative Gradient-Based Projection (IGBP), a novel method for removing non-linear encoded concepts from neural representations. Our method consists of iteratively training neural classifiers to predict a particular attribute we seek to eliminate, followed by a projection of the representation on a hypersurface, such that the classifiers become oblivious to the target attribute. We evaluate the effectiveness of our method on the task of removing gender and race information as sensitive attributes. Our results demonstrate that IGBP is effective in mitigating bias through intrinsic and extrinsic evaluations, with minimal impact on downstream task accuracy.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 27 of its 41 references that are in the map. First posted on arXiv 17 May 2023.

Builds on

27 of 41 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  4. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  5. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  6. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  7. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 659
  8. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 455
  9. 2018Gender Bias in Coreference Resolutioncited by 442
  10. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 349
  11. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  12. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 232
15 more
  1. 2016Demographic Dialectal Variation in Social Media: A Case Study of African-American Englishcited by 196
  2. 2014SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimationcited by 172
  3. 2018Adversarial Removal of Demographic Attributes from Text Datacited by 170
  4. 2017Data Decisions and Theoretical Implications when Adversarially Learning Fair Representationscited by 165
  5. 2017Controllable Invariance through Adversarial Feature Learningcited by 127
  6. 2017Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasmcited by 118
  7. 2022Linear Adversarial Concept Erasurecited by 101
  8. 2019What's in a Name? Reducing Bias in Bios without Access to Protected Attributescited by 51
  9. 2022Kernelized Concept Erasurecited by 48
  10. 2022Probing Classifiers are Unreliable for Concept Removal and Detectioncited by 41
  11. 2020Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigationcited by 36
  12. 2022Gold Doesn't Always Glitter: Spectral Removal of Linear and Nonlinear Guarded Attribute Informationcited by 34
  13. 2022Choose Your Lenses: Flaws in Gender Bias Evaluationcited by 29
  14. 2021Debiasing Methods in Natural Language Understanding Make Bias More Accessiblecited by 17
  15. 2022How Gender Debiasing Affects Internal Model Representations, and Why It Matterscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 12; 2025: 5; 2026: 5.

25 papers.

Sort:

By area:

  1. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  2. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  3. 2023A Survey on Fairness in Large Language Modelscited by 95
  4. 2024Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineeringcited by 63
  5. 2024Spectral Editing of Activations for Large Language Model Alignmentcited by 56
  6. 2025Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuningcited by 39
  7. 2024GenderBias-VL: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probingcited by 31
  8. 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysiscited by 24
  9. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  10. 2025BiasEdit: Debiasing Stereotyped Language Models via Model Editingcited by 18
  11. 2024On the Scaling Laws of Geographical Representation in Language Modelscited by 14
  12. 2024AGR: Age Group fairness Reward for Bias Mitigation in LLMscited by 12
13 more
  1. 2023Tackling Bias in Pre-trained Language Models: Current Trends and Under-represented Societiescited by 11
  2. 2025Precise In-Parameter Concept Erasure in Large Language Modelscited by 11
  3. 2026Spectral Attention Steering for Prompt Highlightingcited by 8
  4. 2024Focus On This, Not That! Steering LLMs with Adaptive Feature Specificationcited by 5
  5. 2024Leveraging Prototypical Representations for Mitigating Social Bias without Demographic Informationcited by 3
  6. 2024The Life Cycle of Large Language Models: A Review of Biases in Educationcited by 3
  7. 2024MultiContrievers: Analysis of Dense Retrieval Representationscited by 1
  8. 2025FALCON: Fine-grained Activation Manipulation by Contrastive Orthogonal Unalignment for Large Language Modelcited by 1
  9. 2025Multimodal Large Language Models for Enhanced Traffic Safety: A Comprehensive Review and Future Trendscited by 1
  10. 2026Understanding Empirical Unlearning with Combinatorial Interpretabilitycited by 0
  11. 2026MANCE: Manifold Aware Concept Erasurecited by 0
  12. 2026Fair on the Surface? Benchmarking Hidden-Output Fairness Gaps in LLM Recommenderscited by 0
  13. 2026Measurement-Gated Provenance Attenuation for Frozen EEG Representationscited by 0