mlsanityData to 7 Oct 2026

Improving Activation Steering in Language Models with Mean-Centring

arXiv 2312.03813 ↗First posted 6 Dec 2023cs.CL cs.AI cs.LGcited by 64

Not on the map. Why?

Abstract (arXiv)

Recent work in activation steering has demonstrated the potential to better control the outputs of Large Language Models (LLMs), but it involves finding steering vectors. This is difficult because engineers do not typically know how features are represented in these models. We seek to address this issue by applying the idea of mean-centring to steering vectors. We find that taking the average of activations associated with a target dataset, and then subtracting the mean of all training activations, results in effective steering vectors. We test this method on a variety of models on natural language tasks by steering away from generating toxic text, and steering the completion of a story towards a target genre. We also apply mean-centring to extract function vectors, more effectively triggering the execution of a range of natural language tasks by a significant margin (compared to previous baselines). This suggests that mean-centring can be used to easily improve the effectiveness of activation steering in a wide range of contexts.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Builds on 8 of its 16 references that are in the map. First posted on arXiv 6 Dec 2023.

Builds on

8 of 16 references.

References · 8

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  4. 2018Deep contextualized word representationscited by 2,285
  5. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  6. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  7. 2023LEACE: Perfect linear concept erasure in closed formcited by 254
  8. 2017All-but-the-Top: Simple and Effective Postprocessing for Word Representationscited by 205

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 28; 2026: 21.

64 papers.

Sort:

By area:

  1. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  2. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  3. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 100
  4. 2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 82
  5. 2024Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categoriescited by 74
  6. 2024Steering Without Side Effects: Improving Post-Deployment Control of Language Modelscited by 44
  7. 2024Robustifying Safety-Aligned Large Language Models through Clean Data Curationcited by 43
  8. 2024Extending Activation Steering to Broad Skills and Multiple Behaviourscited by 36
  9. 2024Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineeringcited by 33
  10. 2025SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Modelscited by 32
  11. 2024Controlling Large Language Model Agents with Entropic Activation Steeringcited by 29
  12. 2025Representation Bending for Large Language Model Safetycited by 28
38 more
  1. 2025SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Modelscited by 28
  2. 2024Tradeoffs Between Alignment and Helpfulness in Language Models with Steering Methodscited by 26
  3. 2025Towards Understanding Distilled Reasoning Models: A Representational Approachcited by 26
  4. 2025To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Modelscited by 26
  5. 2024Beyond Single Concept Vector: Modeling Concept Subspace in LLMs with Gaussian Distributioncited by 24
  6. 2024Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Modelscited by 18
  7. 2024A Causal Explainable Guardrails for Large Language Modelscited by 18
  8. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 18
  9. 2025Enhancing Semantic Consistency of Large Language Models through Model Editing: An Interpretability-Oriented Approachcited by 14
  10. 2025BILLY: Steering Large Language Models via Merging Persona Vectors for Creative Generationcited by 10
  11. 2025Denoising Concept Vectors with Sparse Autoencoders for Improved Language Model Steeringcited by 9
  12. 2025Risk Assessment Framework for Code LLMs via Leveraging Internal Statescited by 8
  13. 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLMcited by 8
  14. 2026Causality is Key for Interpretability Claims to Generalisecited by 8
  15. 2025IF-GUIDE: Influence Function-Guided Detoxification of LLMscited by 7
  16. 2025What Can We Actually Steer? A Multi-Behavior Study of Activation Controlcited by 6
  17. 2026Differential syntactic and semantic encoding in LLMscited by 6
  18. 2026Efficient and accurate steering of Large Language Models through attention-guided feature learningcited by 6
  19. 2025Designing Role Vectors to Improve LLM Inference Behaviourcited by 5
  20. 2025From Emergence to Control: Probing and Modulating Self-Reflection in Language Modelscited by 5
  21. 2025ReCoVeR the Target Language: Language Steering without Sacrificing Task Performancecited by 5
  22. 2025Emergence of psychopathological computations in large language modelscited by 4
  23. 2025Linear Spatial World Models Emerge in Large Language Modelscited by 4
  24. 2025Probing the Robustness of Large Language Models Safety to Latent Perturbationscited by 4
  25. 2025From Concepts to Components: Concept-Agnostic Attention Module Discovery in Transformerscited by 4
  26. 2025Toward a Flexible Framework for Linear Representation Hypothesis Using Maximum Likelihood Estimationcited by 3
  27. 2025LayerCake: Token-Aware Contrastive Decoding within Large Language Model Layerscited by 3
  28. 2026Controllable Value Alignment in Large Language Models through Neuron-Level Editingcited by 3
  29. 2026Concept Heterogeneity-aware Representation Steeringcited by 3
  30. 2026The Cylindrical Representation Hypothesis for Language Model Steeringcited by 3
  31. 2024Data to Defense: The Role of Curation in Customizing LLMs Against Jailbreaking Attackscited by 2
  32. 2025Balancing Stylization and Truth via Disentangled Representation Steeringcited by 2
  33. 2025Who is In Charge? Dissecting Role Conflicts in Instruction Followingcited by 2
  34. 2026Neuron-Level Emotion Control in Speech-Generative Large Audio-Language Modelscited by 2
  35. 2026Therefore I am. I Thinkcited by 2
  36. 2026Decodable but Not Corrected by Fixed Residual-Stream Linear Steering: Evidence from Medical LLM Failure Regimescited by 2
  37. 2026When is Your LLM Steerable?cited by 2
  38. 2026Where Steering Signals Come From: Activation Source Selection in Activation Steeringcited by 2

50 most cited of 64 All 64 (JSON)