mlsanityData to 7 Oct 2026

Improving Instruction-Following in Language Models through Activation Steering

arXiv 2410.12877 ↗First posted 15 Oct 2024cs.CL cs.AI cs.LGcited by 149

Not on the map. Why?

Cited by 20 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The ability to follow instructions is crucial for numerous real-world applications of language models. In pursuit of deeper insights and more powerful capabilities, we derive instruction-specific vector representations from language models and use them to steer models accordingly. These vectors are computed as the difference in activations between inputs with and without instructions, enabling a modular approach to activation steering. We demonstrate how this method can enhance model adherence to constraints such as output format, length, and word inclusion, providing inference-time control over instruction following. Our experiments across four models demonstrate how we can use the activation vectors to guide models to follow constraints even without explicit instructions and to enhance performance when instructions are present. Additionally, we explore the compositionality of activation steering, successfully applying multiple instructions simultaneously. Finally, we demonstrate that steering vectors computed on instruction-tuned models can transfer to improve base models. Our findings demonstrate that activation steering offers a practical and scalable approach for fine-grained control in language generation. Our code and data are available at https://github.com/microsoft/llm-steer-instruct.

Key facts. Cited by 149 papers in the map (data through 7 Oct 2026). Builds on 61 of its 80 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

61 of 80 references.

References · 61

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  5. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  7. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  8. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  9. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  10. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  11. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  12. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
38 more
  1. 2022Editing Models with Task Arithmeticcited by 1,076
  2. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  3. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  4. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  5. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  6. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  7. 2023Large Language Models as Optimizerscited by 830
  8. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  9. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  10. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  11. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  12. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  13. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  14. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  15. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  16. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  17. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  18. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  19. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  20. 2023Language Models Represent Space and Timecited by 306
  21. 2023In-Context Learning Creates Task Vectorscited by 302
  22. 2023Function Vectors in Large Language Modelscited by 292
  23. 2023LEACE: Perfect linear concept erasure in closed formcited by 254
  24. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  25. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
  26. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicitycited by 179
  27. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  28. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 140
  29. 2024Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Trackingcited by 140
  30. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
  31. 2024Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimizationcited by 112
  32. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
  33. 2023Controlled Text Generation with Natural Language Instructionscited by 87
  34. 2023Evaluating Large Language Models on Controlled Generation Taskscited by 73
  35. 2024Spectral Editing of Activations for Large Language Model Alignmentcited by 54
  36. 2024What Makes and Breaks Safety Fine-tuning? A Mechanistic Studycited by 54
  37. 2023Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languagescited by 52
  38. 2024Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representationscited by 45

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 49; 2026: 95.

149 papers.

Sort:

By area:

  1. 2025Generalizing Verifiable Instruction Followingcited by 178
  2. 2025MemOS: A Memory OS for AI Systemcited by 129
  3. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 104
  4. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 89
  5. 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 45
  6. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  7. 2025MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agentscited by 37
  8. 2025Evaluating and Steering Modality Preferences in Multimodal Large Language Modelcited by 33
  9. 2025SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Modelscited by 32
  10. 2026How do LLMs Compute Verbal Confidencecited by 23
  11. 2025Activation Steering for Chain-of-Thought Compressioncited by 22
  12. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
38 more
  1. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 18
  2. 2025Spotlight Your Instructions: Instruction-following with Dynamic Attention Steeringcited by 18
  3. 2025The Rogue Scalpel: Activation Steering Compromises LLM Safetycited by 18
  4. 2025Transferring Linear Features Across Language Models With Model Stitchingcited by 17
  5. 2025Towards High Data Efficiency in Reinforcement Learning with Verifiable Rewardcited by 16
  6. 2025Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Racecited by 15
  7. 2026ODESteer: A Unified ODE-Based Steering Framework for LLM Alignmentcited by 15
  8. 2024All or None: Identifiable Linear Properties of Next-token Predictors in Language Modelingcited by 14
  9. 2025SAKE: Steering Activations for Knowledge Editingcited by 12
  10. 2025Do different prompting methods yield a common task representation in language models?cited by 9
  11. 2025Denoising Concept Vectors with Sparse Autoencoders for Improved Language Model Steeringcited by 9
  12. 2025SafeSteer: Interpretable Safety Steering with Refusal-Evasion in LLMscited by 9
  13. 2025Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoderscited by 9
  14. 2026Causal Evidence that Language Models use Confidence to Drive Behaviorcited by 9
  15. 2026Analysing the Safety Pitfalls of Steering Vectorscited by 9
  16. 2025Guiding Giants: Lightweight Controllers for Weighted Activation Steering in LLMscited by 8
  17. 2025KV Cache Steering for Controlling Frozen LLMscited by 8
  18. 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLMcited by 8
  19. 2026Steering Externalities: Benign Activation Steering Unintentionally Increases Jailbreak Risk for Large Language Modelscited by 8
  20. 2025EasyEdit2: An Easy-to-use Steering Framework for Editing Large Language Modelscited by 7
  21. 2025Steering When Necessary: Flexible Steering Large Language Models with Backtrackingcited by 7
  22. 2025Improving Generative Behavior Cloning via Self-Guidance and Adaptive Chunkingcited by 7
  23. 2025Learning Plug-and-play Memory for Guiding Video Diffusion Modelscited by 7
  24. 2026Linear representations in language models can change dramatically over a conversationcited by 7
  25. 2024Attention Speaks Volumes: Localizing and Mitigating Bias in Language Modelscited by 6
  26. 2025Linear Representation Transferability Hypothesis: Leveraging Small Models to Steer Large Modelscited by 6
  27. 2025Enhancing Cross-task Transfer of Large Language Models via Activation Steeringcited by 6
  28. 2025Revisiting the Reliability of Language Models in Instruction-Followingcited by 6
  29. 2026Efficient and accurate steering of Large Language Models through attention-guided feature learningcited by 6
  30. 2026Fine-Grained Activation Steering: Steering Less, Achieving Morecited by 6
  31. 2026DLM-Scope: Mechanistic Interpretability of Diffusion Language Models via Sparse Autoencoderscited by 6
  32. 2026Spectral Attention Steering for Prompt Highlightingcited by 6
  33. 2025ReCoVeR the Target Language: Language Steering without Sacrificing Task Performancecited by 5
  34. 2025DISCO: Disentangled Communication Steering for Large Language Modelscited by 5
  35. 2026Compositional Steering of Large Language Models with Steering Tokenscited by 5
  36. 2026EmoShift: Lightweight Activation Steering for Enhanced Emotion-Aware Speech Synthesiscited by 5
  37. 2025ExpertSteer: Intervening in LLMs through Expert Knowledgecited by 4
  38. 2025When Instructions Multiply: Measuring and Estimating LLM Capabilities of Multiple Instructions Followingcited by 4

50 most cited of 149 All 149 (JSON)