mlsanityData to 7 Oct 2026

Steering Language Models With Activation Engineering

arXiv 2308.10248 ↗First posted 20 Aug 2023cs.CL cs.LGcited by 824

Not on the map. Why?

Cited by 105 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Prompt engineering and finetuning aim to maximize language model performance on a given metric (like toxicity reduction). However, these methods do not fully elicit a model's capabilities. To reduce this gap, we introduce activation engineering: the inference-time modification of activations in order to control (or steer) model outputs. Specifically, we introduce the Activation Addition (ActAdd) technique, which contrasts the intermediate activations on prompt pairs (such as "Love" versus "Hate") to compute a steering vector (Subramani et al. 2022). By tactically adding in e.g. the "Love" - "Hate" steering vector during the forward pass, we achieve SOTA on negative-to-positive sentiment shift and detoxification using models including LLaMA-3 and OPT. ActAdd yields inference-time control over high-level output properties (like topic and sentiment) while preserving performance on off-target tasks. ActAdd is lightweight: it does not require any machine optimization and works with a single pair of data points, which enables rapid iteration over steering. ActAdd demonstrates the power of activation engineering.

Key facts. Cited by 824 papers in the map (data through 7 Oct 2026). Builds on 31 of its 63 references that are in the map. First posted on arXiv 20 Aug 2023.

Builds on

31 of 63 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  7. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  8. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  9. 2019Language Models as Knowledge Bases?cited by 1,922
  10. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  11. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  12. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
19 more
  1. 2019Are Sixteen Heads Really Better than One?cited by 803
  2. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  3. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  4. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  5. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  6. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 374
  7. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  8. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  9. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
  10. 2018Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfercited by 191
  11. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoningcited by 169
  12. 2020Deep Learning for Text Style Transfer: A Surveycited by 161
  13. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
  14. 2019Bias Correction of Learned Generative Models using Likelihood-Free Importance Weightingcited by 70
  15. 2021Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Promptscited by 64
  16. 2023PREADD: Prefix-Adaptive Decoding for Controlled Text Generationcited by 29
  17. 2022Controllable Text Generation via Probability Density Estimation in the Latent Spacecited by 27
  18. 2020Raiders of the Lost Kek: 3.5 Years of Augmented 4chan Posts from the Politically Incorrect Boardcited by 24
  19. 2017Trainable Greedy Decoding for Neural Machine Translationcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 51; 2025: 193; 2026: 570.

824 papers.

Sort:

By area:

  1. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  2. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  3. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  4. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  5. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  6. 2024ReFT: Representation Finetuning for Language Modelscited by 186
  7. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMscited by 174
  8. 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 153
  9. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 145
  10. 2025MemOS: A Memory OS for AI Systemcited by 129
  11. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  12. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 122
38 more
  1. 2024Aligner: Efficient Alignment by Learning to Correctcited by 96
  2. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 89
  3. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  4. 2024InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidancecited by 78
  5. 2025Persona Features Control Emergent Misalignmentcited by 76
  6. 2025LLMs Encode Harmfulness and Refusal Separatelycited by 75
  7. 2024Test-Time Model Adaptation with Only Forward Passescited by 74
  8. 2025SAEs Are Good for Steering -- If You Select the Right Featurescited by 74
  9. 2025Understanding Reasoning in Thinking Language Models via Steering Vectorscited by 69
  10. 2025MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Modelscited by 68
  11. 2025Convergent Linear Representations of Emergent Misalignmentcited by 66
  12. 2024Benchmarking Distributional Alignment of Large Language Modelscited by 58
  13. 2024Style Vectors for Steering Generative Large Language Modelcited by 56
  14. 2024Spectral Editing of Activations for Large Language Model Alignmentcited by 54
  15. 2023RECALL: A Benchmark for LLMs Robustness against External Counterfactual Knowledgecited by 51
  16. 2023Codebook Features: Sparse and Discrete Interpretability for Neural Networkscited by 45
  17. 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 45
  18. 2024Steering Without Side Effects: Improving Post-Deployment Control of Language Modelscited by 44
  19. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  20. 2025Multi-Attribute Steering of Language Models via Targeted Interventioncited by 44
  21. 2024Designing a Dashboard for Transparency and Control of Conversational AIcited by 43
  22. 2025Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainerscited by 41
  23. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  24. 2024Dissecting Language Models: Machine Unlearning via Selective Pruningcited by 38
  25. 2024Advancing Parameter Efficiency in Fine-tuning via Representation Editingcited by 37
  26. 2025Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activationscited by 37
  27. 2024Extending Activation Steering to Broad Skills and Multiple Behaviourscited by 36
  28. 2024Activation Scaling for Steering and Interpreting Language Modelscited by 33
  29. 2025Interpretable Steering of Large Language Models with Feature Guided Activation Additionscited by 32
  30. 2025Mitigating Overthinking in Large Reasoning Models via Manifold Steeringcited by 32
  31. 2025Probing and Steering Evaluation Awareness of Language Modelscited by 31
  32. 2025Linear Representations of Political Perspective Emerge in Large Language Modelscited by 30
  33. 2025Pixels Versus Priors: Controlling Knowledge Priors in Vision-Language Models through Visual Counterfactscited by 30
  34. 2024Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Modelscited by 29
  35. 2024Controlling Large Language Model Agents with Entropic Activation Steeringcited by 29
  36. 2024No Two Devils Alike: Unveiling Distinct Mechanisms of Fine-tuning Attackscited by 28
  37. 2025Representation Bending for Large Language Model Safetycited by 28
  38. 2025SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Modelscited by 28

50 most cited of 824 All 824 (JSON)