mlsanityData to 14 Oct 2026

Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineering

arXiv 2410.16314 ↗First posted 9 Oct 2024cs.NE cs.LGcited by 39

On the map: Mechanistic interpretability → Activation steering methods

Abstract (arXiv)

Large language models have transformed AI, yet reliably controlling their outputs remains a challenge. This paper explores activation engineering, where outputs of pre-trained LLMs are controlled by manipulating their activations at inference time. Unlike traditional methods using a single steering vector, we introduce conceptors - mathematical constructs that represent sets of activation vectors as ellipsoidal regions. Conceptors act as soft projection matrices and offer more precise control over complex activation patterns. Our experiments demonstrate that conceptors outperform traditional methods across multiple steering tasks. We further use Boolean operations on conceptors for combined steering goals that empirically outperform additively combining steering vectors on a set of tasks. These results highlight conceptors as a promising tool for more effective steering of LLMs. Our code is available on github.com/jorispos/conceptorsteering.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 15 of its 20 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

15 of 20 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Extending Activation Steering to Broad Skills and Multiple Behaviours9 Mar
  2. 2023Steering Llama 2 via Contrastive Activation Addition9 Dec
  3. 2023Improving Activation Steering in Language Models with Mean-Centring6 Dec

Other references · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  4. 2016Concrete Problems in AI Safetycited by 1,642
  5. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  6. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  7. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  8. 2023Function Vectors in Large Language Modelscited by 308
  9. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 128
  10. 2014Controlling Recurrent Neural Networks by Conceptorscited by 16
  11. 2022Conceptor-Aided Debiasing of Large Language Modelscited by 5
  12. 2014Conceptors: an easy introductioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 30.

39 papers.

Sort:

By area:

  1. 2025To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Modelscited by 29
  2. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  3. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 24
  4. 2026Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Modelscited by 15
  5. 2025KV Cache Steering for Controlling Frozen LLMscited by 11
  6. 2026Why Steering Works: Toward a Unified View of Language Model Parameter Dynamicscited by 9
  7. 2025SteerVLM: Robust Model Control through Lightweight Activation Steering for Vision Language Modelscited by 8
  8. 2026From Weights to Activations: Is Steering the Next Frontier of Adaptation?cited by 8
  9. 2026Fine-Grained Activation Steering: Steering Less, Achieving Morecited by 7
  10. 2026Enhancing Instruction Following of LLMs via Activation Steering with Dynamic Rejectioncited by 6
  11. 2026Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventionscited by 6
  12. 2025A Comparative Analysis of Sparse Autoencoder and Activation Difference in Language Model Steeringcited by 4
27 more
  1. 2025Toward a Flexible Framework for Linear Representation Hypothesis Using Maximum Likelihood Estimationcited by 3
  2. 2026Conceptors for Semantic Steeringcited by 3
  3. 2026Contrastive Conceptor Activation Steering (COAST): Unlocking Vision-Language-Action Models through Hidden Statescited by 3
  4. 2026Where Steering Signals Come From: Activation Source Selection in Activation Steeringcited by 3
  5. 2026Exploring Diverse Generation Paths via Inference-time Stiefel Activation Steeringcited by 2
  6. 2026Activation Surgery: Jailbreaking White-box LLMs without Touching the Promptcited by 2
  7. 2026Decomposing how prompting steers behaviorcited by 2
  8. 2025Fusion Steering: Prompt-Specific Activation Controlcited by 1
  9. 2025IA2: Alignment with ICL Activations Improves Supervised Fine-Tuningcited by 1
  10. 2026Steerability of Instrumental-Convergence Tendencies in LLMscited by 1
  11. 2025Concept Attractors in LLMs and their Applicationscited by 1
  12. 2026Prometheus Mind: Retrofitting Memory to Frozen Language Modelscited by 1
  13. 2026How Do Answer Tokens Read Reasoning Traces? Self-Reading Patterns in Thinking LLMs for Quantitative Reasoningcited by 1
  14. 2026Spillover-Aware Multi-Value Steering for Pluralistic LLM Alignmentcited by 1
  15. 2026Focusing Condition: Inference-Time Self-Contrastive Steering Elicits Better Conditional Text Embeddings in LLMscited by 1
  16. 2026CRAFT: Causal Responsibility and Failure Tracing in Medical Vision Language Modelscited by 1
  17. 2026α-TCAV: A Unified Framework for Testing with Concept Activation Vectorscited by 0
  18. 2026Universal Boosts, Specific Suppressors: Sparse Autoencoder Steering of Medical Vision-Language Modelscited by 0
  19. 2026Temporal Preference Concepts and their Functions in a Large Language Modelcited by 0
  20. 2026Probabilistic Concept-Aware Steering for Trustworthy LLM Inferencecited by 0
  21. 2026Latent-IM: Latent Interaction Management for Speech LLMscited by 0
  22. 2026PCA-guided Activation Scaling for Monotonic Bidirectional Control over LLM Sycophancycited by 0
  23. 2026Abliteration Mitigation via Refusal Aliasescited by 0
  24. 2026Capturing In-Context Learning Dynamics with Task Operatorscited by 0
  25. 2026One Token Can Be Enough: Bridging Prompting and Activation Steering with Prefix Steeringcited by 0
  26. 2026The Sharp Directions Are Against You: Curvature Analysis of Activation Steeringcited by 0
  27. 2026When a Conceptor Complement Becomes Gain: Operator Geometry and Behavioral Attribution in Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Controlling Large Language Model Agents with Entropic Activation Steering
  2. 2024Improving Instruction-Following in Language Models through Activation Steering
  3. 2024Activation Scaling for Steering and Interpreting Language Models
  4. 2025Steering Large Language Model Activations in Sparse Spaces
  5. 2026Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs
  6. 2025Interpretable Steering of Large Language Models with Feature Guided Activation Additions
  7. 2026COLD-Steer: Steering Large Language Models via In-Context One-step Learning Dynamics
  8. 2026Mechanistic Indicators of Steering Effectiveness in Large Language Models