mlsanityData to 7 Oct 2026

Unsupervised Concept Vector Extraction for Bias Control in LLMs

arXiv 2502.19721 ↗First posted 27 Feb 2025cs.CL cs.CYcited by 7

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are known to perpetuate stereotypes and exhibit biases. Various strategies have been proposed to mitigate these biases, but most work studies biases as a black-box problem without considering how concepts are represented within the model. We adapt techniques from representation engineering to study how the concept of "gender" is represented within LLMs. We introduce a new method that extracts concept representations via probability weighting without labeled data and efficiently selects a steering vector for measuring and manipulating the model's representation. We develop a projection-based method that enables precise steering of model predictions and demonstrate its effectiveness in mitigating gender bias in LLMs and show that it also generalizes to racial bias. Our code is available at: https://github.com/hannahxchen/gender-bias-steering

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 21 of its 38 references that are in the map. First posted on arXiv 27 Feb 2025.

Builds on

21 of 38 references.

References · 21

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  3. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  5. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  6. 2018Gender Bias in Coreference Resolutioncited by 438
  7. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  8. 2017Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairnesscited by 381
  9. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 204
  10. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  11. 2023Linear Representations of Sentiment in Large Language Modelscited by 157
  12. 2020NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraintscited by 146
9 more
  1. 2019Toward Gender-Inclusive Coreference Resolutioncited by 84
  2. 2022Theories of "Gender" in NLP Bias Researchcited by 62
  3. 2024Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discriminationcited by 55
  4. 2020Investigating African-American Vernacular English in Transformer-Based Text Generationcited by 52
  5. 2024Evaluating Gender Bias in Large Language Models via Chain-of-Thought Promptingcited by 49
  6. 2023A Trip Towards Fairness: Bias and De-Biasing in Large Language Modelscited by 32
  7. 2023Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Modelcited by 22
  8. 2024BiasAlert: A Plug-and-play Tool for Social Bias Detection in LLMscited by 21
  9. 2025Rejected Dialects: Biases Against African American Language in Reward Modelscited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Controlcited by 14
  2. 2025Refusal Steering: Fine-grained Control over LLM Refusal Behaviour for Sensitive Topicscited by 5
  3. 2026Neuron-Level Interventions for Gendered and Gender-Neutral Generation in Language Modelscited by 2
  4. 2025Understanding Formal Reasoning Failures in LLMs as Abstract Interpreterscited by 1
  5. 2026NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM Agentscited by 1
  6. 2026White-Box Sensitivity Auditing with Steering Vectorscited by 0
  7. 2026When Do Model Internals Help? Exploring the Role of Representation Engineering in LLM Safetycited by 0