# Steering Language Models With Activation Engineering

arXiv: https://arxiv.org/abs/2308.10248 · HTML: https://mlsanity.com/paper/2308.10248/

## Key facts

- Cited by 922 papers in the map (data through 14 Oct 2026).
- Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability.
- Builds on 31 of its 63 references that are in the map.
- First posted on arXiv 20 Aug 2023.

## Placed in

- Mechanistic interpretability / [Activation steering methods](https://mlsanity.com/mechinterp/activation-steering-methods/)

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [FUDGE: Controlled Text Generation With Future Discriminators](https://mlsanity.com/paper/2104.05218/) (2021)
- [Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP](https://mlsanity.com/paper/2103.00453/) (2021)
- [Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models](https://mlsanity.com/paper/2301.04213/) (2023)
- [Extracting Latent Steering Vectors from Pretrained Language Models](https://mlsanity.com/paper/2205.05124/) (2022)
- [Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://mlsanity.com/paper/1804.06437/) (2018)
- [The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning](https://mlsanity.com/paper/2205.03401/) (2022)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Models](https://mlsanity.com/paper/2201.05337/) (2022)
- [Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting](https://mlsanity.com/paper/1906.09531/) (2019)
- [Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts](https://mlsanity.com/paper/2112.08348/) (2021)
- [PREADD: Prefix-Adaptive Decoding for Controlled Text Generation](https://mlsanity.com/paper/2307.03214/) (2023)
- [Controllable Text Generation via Probability Density Estimation in the Latent Space](https://mlsanity.com/paper/2212.08307/) (2022)
- [Raiders of the Lost Kek: 3.5 Years of Augmented 4chan Posts from the Politically Incorrect Board](https://mlsanity.com/paper/2001.07487/) (2020)

## Built on by (most cited first, 30 of 922)

- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering](https://mlsanity.com/paper/2311.06668/) (2023)
- [AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders](https://mlsanity.com/paper/2501.17148/) (2025)
- [ReFT: Representation Finetuning for Language Models](https://mlsanity.com/paper/2404.03592/) (2024)
- [Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet](https://mlsanity.com/paper/2605.29358/) (2026)
- [Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs](https://mlsanity.com/paper/2407.21771/) (2024)
- [Eight Methods to Evaluate Robust Unlearning in LLMs](https://mlsanity.com/paper/2402.16835/) (2024)
- [LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet](https://mlsanity.com/paper/2408.15221/) (2024)
- [MemOS: A Memory OS for AI System](https://mlsanity.com/paper/2507.03724/) (2025)
- [Black-Box Access is Insufficient for Rigorous AI Audits](https://mlsanity.com/paper/2401.14446/) (2024)
- [Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models](https://mlsanity.com/paper/2405.16833/) (2024)
- [Aligner: Efficient Alignment by Learning to Correct](https://mlsanity.com/paper/2402.02416/) (2024)
- [Steering Language Model Refusal with Sparse Autoencoders](https://mlsanity.com/paper/2411.11296/) (2024)
- [LLMs Encode Harmfulness and Refusal Separately](https://mlsanity.com/paper/2507.11878/) (2025)
- [Persona Features Control Emergent Misalignment](https://mlsanity.com/paper/2506.19823/) (2025)
- [A Primer on the Inner Workings of Transformer-based Language Models](https://mlsanity.com/paper/2405.00208/) (2024)
- [InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance](https://mlsanity.com/paper/2401.11206/) (2024)
- [SAEs Are Good for Steering -- If You Select the Right Features](https://mlsanity.com/paper/2505.20063/) (2025)
- [Verbalizable Representations Form a Global Workspace in Language Models](https://mlsanity.com/paper/2607.15495/) (2026)
- [Test-Time Model Adaptation with Only Forward Passes](https://mlsanity.com/paper/2404.01650/) (2024)
- [MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models](https://mlsanity.com/paper/2505.22101/) (2025)
- [Understanding Reasoning in Thinking Language Models via Steering Vectors](https://mlsanity.com/paper/2506.18167/) (2025)
- [Convergent Linear Representations of Emergent Misalignment](https://mlsanity.com/paper/2506.11618/) (2025)
- [Benchmarking Distributional Alignment of Large Language Models](https://mlsanity.com/paper/2411.05403/) (2024)
- [Emotion Concepts and their Function in a Large Language Model](https://mlsanity.com/paper/2604.07729/) (2026)
- [Style Vectors for Steering Generative Large Language Model](https://mlsanity.com/paper/2402.01618/) (2024)
- [Spectral Editing of Activations for Large Language Model Alignment](https://mlsanity.com/paper/2405.09719/) (2024)
- [Angular Steering: Behavior Control via Rotation in Activation Space](https://mlsanity.com/paper/2510.26243/) (2025)

## Abstract (arXiv)

Prompt engineering and finetuning aim to maximize language model performance on a given metric (like toxicity reduction). However, these methods do not fully elicit a model's capabilities. To reduce this gap, we introduce activation engineering: the inference-time modification of activations in order to control (or steer) model outputs. Specifically, we introduce the Activation Addition (ActAdd) technique, which contrasts the intermediate activations on prompt pairs (such as "Love" versus "Hate") to compute a steering vector (Subramani et al. 2022). By tactically adding in e.g. the "Love" - "Hate" steering vector during the forward pass, we achieve SOTA on negative-to-positive sentiment shift and detoxification using models including LLaMA-3 and OPT. ActAdd yields inference-time control over high-level output properties (like topic and sentiment) while preserving performance on off-target tasks. ActAdd is lightweight: it does not require any machine optimization and works with a single pair of data points, which enables rapid iteration over steering. ActAdd demonstrates the power of activation engineering.