mlsanityData to 14 Oct 2026

Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Race

arXiv 2506.00253 ↗First posted 30 May 2025cs.CL cs.AI cs.CYcited by 15

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

also in 1 more
Abstract (arXiv)

Although value-aligned language models (LMs) appear unbiased in explicit bias evaluations, they often exhibit stereotypes in implicit word association tasks, raising concerns about their fair usage. We investigate the mechanisms behind this discrepancy and find that alignment surprisingly amplifies implicit bias in model outputs. Specifically, we show that aligned LMs, unlike their unaligned counterparts, overlook racial concepts in early internal representations when the context is ambiguous. Not representing race likely fails to activate safety guardrails, leading to unintended biases. Inspired by this insight, we propose a new bias mitigation strategy that works by incentivizing the representation of racial concepts in the early model layers. In contrast to conventional mitigation methods of machine unlearning, our interventions find that steering the model to be more aware of racial concepts effectively mitigates implicit bias. Similar to race blindness in humans, ignoring racial nuances can inadvertently perpetuate subtle biases in LMs.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 34 of its 59 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

34 of 59 references.

References · 34

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  4. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  5. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  6. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  7. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  8. 2023Steering Language Models With Activation Engineeringcited by 922
  9. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  10. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  11. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  12. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
22 more
  1. 2021Causal Abstractions of Neural Networkscited by 422
  2. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
  3. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  4. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  5. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 218
  6. 2024How to use and interpret activation patchingcited by 214
  7. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 160
  8. 2023The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networkscited by 156
  9. 2024Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescuecited by 103
  10. 2023Clinical Camel: An Open Expert-Level Medical Language Model with Dialogue-Based Knowledge Encodingcited by 71
  11. 2024SelfIE: Self-Interpretation of Large Language Model Embeddingscited by 67
  12. 2024Uncovering Safety Risks of Large Language Models through Concept Activation Vectorcited by 67
  13. 2024Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNscited by 65
  14. 2024Auxiliary task demands mask the capabilities of smaller language modelscited by 54
  15. 2024Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspectivecited by 48
  16. 2023Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domaincited by 38
  17. 2025IssueBench: Millions of Realistic Prompts for Measuring Issue Bias in LLM Writing Assistancecited by 26
  18. 2024Investigating Implicit Bias in Large Language Models: A Large-Scale Study of Over 50 LLMscited by 25
  19. 2023Exploring the impact of low-rank adaptation on the performance, efficiency, and regularization of RLHFcited by 20
  20. 2023Generative Models as a Complex Systems Science: How can we make sense of large language model behavior?cited by 19
  21. 2024Mitigating Social Biases in Language Models through Unlearningcited by 7
  22. 2023SuryaKiran at MEDIQA-Sum 2023: Leveraging LoRA for Clinical Dialogue Summarizationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 10.

15 papers.

Sort:

By area:

  1. 2026Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Controlcited by 16
  2. 2026Different Demographic Cues Yield Inconsistent Conclusions About LLM Personalization and Biascited by 4
  3. 2026Self-Debias: Self-correcting for Debiasing Large Language Modelscited by 3
  4. 2025Large Language Models Develop Novel Social Biases Through Adaptive Explorationcited by 2
  5. 2025EMBRACE: Shaping Inclusive Opinion Representation by Aligning Implicit Conversations with Social Normscited by 1
  6. 2025Evaluating Implicit Biases in LLM Reasoning through Logic Grid Puzzlescited by 1
  7. 2025Measuring Mechanistic Independence: Can Bias Be Removed Without Erasing Demographics?cited by 1
  8. 2026Identity-Robust Language Model Generation via Content Integrity Preservationcited by 1
  9. 2026Intra-Fairness Dynamics: The Bias Spillover Effect in Targeted LLM Alignmentcited by 1
  10. 2026Common Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMscited by 1
  11. 2025When Harmless Words Harm: A New Threat to LLM Safety via Conceptual Triggerscited by 0
  12. 2026Bias Dynamics in BabyLMs: Towards a Compute-Efficient Sandbox for Democratising Pre-Training Debiasingcited by 0
3 more
  1. 2026Sympathetic Framing: Evaluating AI Alignment across Sociodemographic Groupscited by 0
  2. 2026Fair on the Surface? Benchmarking Hidden-Output Fairness Gaps in LLM Recommenderscited by 0
  3. 2026Proxy reliance in large language model decisions is uncalibrated to predictive evidencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Alignment Reduces Expressed but Not Encoded Gender Bias: A Unified Framework and Study
  2. 2024The Root Shapes the Fruit: On the Persistence of Gender-Exclusive Harms in Aligned Language Models
  3. 2025Explicit vs. Implicit: Investigating Social Bias in Large Language Models through Self-Reflection
  4. 2026ImplicitBBQ: Benchmarking Implicit Bias in Large Language Models through Characteristic Based Cues
  5. 2024BiasDora: Exploring Hidden Biased Associations in Vision-Language Models
  6. 2025Mitigating Biases in Language Models via Bias Unlearning
  7. 2025Implicit Bias in LLMs: A Survey
  8. 2026The Wrong Kind of Right: Quantifying and Localizing Misfired Alignment in LLMs