mlsanityData to 7 Oct 2026

Steering Llama 2 via Contrastive Activation Addition

arXiv 2312.06681 ↗First posted 9 Dec 2023cs.CL cs.AI cs.LGcited by 1,009

Not on the map. Why?

Cited by 130 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce Contrastive Activation Addition (CAA), an innovative method for steering language models by modifying their activations during forward passes. CAA computes "steering vectors" by averaging the difference in residual stream activations between pairs of positive and negative examples of a particular behavior, such as factual versus hallucinatory responses. During inference, these steering vectors are added at all token positions after the user's prompt with either a positive or negative coefficient, allowing precise control over the degree of the targeted behavior. We evaluate CAA's effectiveness on Llama 2 Chat using multiple-choice behavioral question datasets and open-ended generation tasks. We demonstrate that CAA significantly alters model behavior, is effective over and on top of traditional methods like finetuning and system prompt design, and minimally reduces capabilities. Moreover, we gain deeper insights into CAA's mechanisms by employing various activation space interpretation methods. CAA accurately steers model outputs and sheds light on how high-level concepts are represented in Large Language Models (LLMs).

Key facts. Cited by 1,009 papers in the map (data through 7 Oct 2026). Builds on 19 of its 71 references that are in the map. First posted on arXiv 9 Dec 2023.

Builds on

19 of 71 references.

References · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  8. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  9. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  10. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  11. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  12. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
7 more
  1. 2023Steering Language Models With Activation Engineeringcited by 824
  2. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  3. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
  4. 2023Linear Representations of Sentiment in Large Language Modelscited by 157
  5. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 140
  6. 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 130
  7. 2023Demystifying Embedding Spaces using Large Language Modelscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 87; 2025: 296; 2026: 626.

1,009 papers.

Sort:

By area:

  1. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  2. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 246
  3. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  4. 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 153
  5. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 149
  6. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 145
  7. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 130
  8. 2025MemOS: A Memory OS for AI Systemcited by 129
  9. 2024Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimizationcited by 112
  10. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 104
  11. 2025LLM Social Simulations Are a Promising Research Methodcited by 97
  12. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 89
38 more
  1. 2024Improving Steering Vectors by Targeting Sparse Autoencoder Featurescited by 86
  2. 2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 82
  3. 2025Persona Features Control Emergent Misalignmentcited by 76
  4. 2025Do Multilingual LLMs Think In English?cited by 75
  5. 2025LLMs Encode Harmfulness and Refusal Separatelycited by 75
  6. 2025SAEs Are Good for Steering -- If You Select the Right Featurescited by 74
  7. 2024The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalitiescited by 69
  8. 2025Understanding Reasoning in Thinking Language Models via Steering Vectorscited by 69
  9. 2025Convergent Linear Representations of Emergent Misalignmentcited by 66
  10. 2024Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectorscited by 61
  11. 2024Emergent World Models and Latent Variable Estimation in Chess-Playing Language Modelscited by 60
  12. 2024Robust LLM safeguarding via refusal feature adversarial trainingcited by 60
  13. 2024Controlling Language and Diffusion Models by Transporting Activationscited by 59
  14. 2024Elements of World Knowledge (EWoK): A Cognition-Inspired Framework for Evaluating Basic World Knowledge in Language Modelscited by 55
  15. 2024Spectral Editing of Activations for Large Language Model Alignmentcited by 54
  16. 2024From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuningcited by 51
  17. 2025Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoningcited by 50
  18. 2024ICLR: In-Context Learning of Representationscited by 49
  19. 2024Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablationcited by 48
  20. 2025Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinationscited by 48
  21. 2025Understanding Aha Moments: from External Observations to Internal Mechanismscited by 48
  22. 2025When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Modelscited by 46
  23. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 45
  24. 2024Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Modelscited by 45
  25. 2024SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steeringcited by 45
  26. 2025AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraintcited by 45
  27. 2025Detecting High-Stakes Interactions with Activation Probescited by 45
  28. 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 45
  29. 2024Steering Without Side Effects: Improving Post-Deployment Control of Language Modelscited by 44
  30. 2024Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreakscited by 44
  31. 2025Multi-Attribute Steering of Language Models via Targeted Interventioncited by 44
  32. 2024Interpreting Attention Layer Outputs with Sparse Autoencoderscited by 43
  33. 2025Learning without training: The implicit dynamics of in-context learningcited by 41
  34. 2025Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainerscited by 41
  35. 2024Who's asking? User personas and the mechanics of latent misalignmentcited by 40
  36. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  37. 2025A Unified Understanding and Evaluation of Steering Methodscited by 40
  38. 2025Enhancing Automated Interpretability with Output-Centric Feature Descriptionscited by 39

50 most cited of 1,009 All 1,009 (JSON)