mlsanityData to 14 Oct 2026

Shifting Perspectives: Steering Vectors for Robust Bias Mitigation in LLMs

arXiv 2503.05371 ↗First posted 7 Mar 2025cs.LG cs.AI cs.CLcited by 14

On the map: Mechanistic interpretability → Probing deception, bias and beliefs

Abstract (arXiv)

We present a novel approach to bias mitigation in large language models (LLMs) by applying steering vectors to modify model activations in forward passes. We compute 8 steering vectors, each corresponding to a different social bias axis, such as age, gender, or race, on a training subset of the BBQ dataset and compare the effectiveness of these to 3 additional bias mitigation methods across 4 datasets. When optimized on the BBQ dataset, our individually tuned steering vectors achieve average improvements of 12.8% on BBQ, 8.3% on CLEAR-Bias, and 1% on StereoSet, and show improvements over prompting and Self-Debias in all cases, and improvements over fine-tuning in 12 out of 17 evaluations. In addition, steering vectors showed the lowest impact on MMLU scores of the four bias mitigation methods tested. The work presents the first systematic investigation of steering vectors for bias mitigation, and we demonstrate that they are a powerful and computationally efficient strategy for reducing bias in LLMs, with broader implications for enhancing AI safety.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Probing deception, bias and beliefs, one of 13 directions in Mechanistic interpretability. Builds on 30 of its 42 references that are in the map. First posted on arXiv 7 Mar 2025.

Builds on

30 of 42 references.

References · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  7. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  8. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  9. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  10. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  11. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  12. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
18 more
  1. 2023Steering Language Models With Activation Engineeringcited by 922
  2. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  3. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  4. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  5. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  6. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  7. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 455
  8. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 337
  9. 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 224
  10. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 207
  11. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  12. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 199
  13. 2020Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biasescited by 167
  14. 2024The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performancecited by 100
  15. 2023Do Models Really Learn to Follow Instructions? An Empirical Study of Instruction Tuningcited by 49
  16. 2024Social Bias Evaluation for Large Language Models Requires Prompt Variationscited by 40
  17. 2023Fake Alignment: Are LLMs Really Aligned Well?cited by 39
  18. 2024IndiBias: A Benchmark Dataset to Measure Social Biases in Language Models for Indian Contextcited by 32

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 13.

14 papers.

Sort:

By area:

  1. 2026CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challengescited by 9
  2. 2026Steering at the Source: Style Modulation Heads for Robust Persona Controlcited by 5
  3. 2026Bias Beyond Borders: Political Ideology Evaluation and Steering in Multilingual LLMscited by 4
  4. 2026Cultural Value Alignment Via Latent Activation Steering in Large Language Modelscited by 3
  5. 2025Fairness Evaluation and Inference Level Mitigation in LLMscited by 2
  6. 2026ConFit v3: Improving Resume-Job Matching with LLM-based Re-Rankingcited by 2
  7. 2026Scenario-based Probing and Steering Cultural Values in Large Language Models--Extended Versioncited by 2
  8. 2026Routing Sensitivity Without Controllability: A Diagnostic Study of Fairness in MoE Language Modelscited by 1
  9. 2026Position: Logical Soundness is not a Reliable Criterion for Neurosymbolic Fact-Checking with LLMscited by 1
  10. 2026Traces of Social Competence in Large Language Modelscited by 0
  11. 2026Analysing Self-Harm Representations in Language Models: a Cross-Architecture Studycited by 0
  12. 2026Inference-Time Policy Alignment for Fair Reinforcement Learningcited by 0
2 more
  1. 2026Safety Cost of Steering Vectors Is Separable and Reduciblecited by 0
  2. 2026Beyond Bias Scores: Answerability Confounds in Activation Steering with Gradient-Constrained Direction Design (GCD)cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Activation Steering for Bias Mitigation: An Interpretable Approach to Safer LLMs
  2. 2025Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judge
  3. 2025DSO: Direct Steering Optimization for Bias Mitigation
  4. 2025Unsupervised Concept Vector Extraction for Bias Control in LLMs
  5. 2025Breaking the Mirror: Activation-Based Mitigation of Self-Preference in LLM Evaluators
  6. 2025Bias Testing and Mitigation in Black Box LLMs using Metamorphic Relations
  7. 2026Latent space bias directions in LLMs capture confidence, not fairness
  8. 2024Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Models