mlsanityData to 7 Oct 2026

Multi-property Steering of Large Language Models with Dynamic Activation Composition

arXiv 2406.17563 ↗First posted 25 Jun 2024cs.CL cs.AI cs.LGcited by 35

Not on the map. Why?

Abstract (arXiv)

Activation steering methods were shown to be effective in conditioning language model generation by additively intervening over models' intermediate representations. However, the evaluation of these techniques has so far been limited to single conditioning properties and synthetic settings. In this work, we conduct a comprehensive evaluation of various activation steering strategies, highlighting the property-dependent nature of optimal parameters to ensure a robust effect throughout generation. To address this issue, we propose Dynamic Activation Composition, an information-theoretic approach to modulate the steering intensity of one or more properties throughout generation. Our experiments on multi-property steering show that our method successfully maintains high conditioning while minimizing the impact of conditioning on generation fluency.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 31 of its 49 references that are in the map. First posted on arXiv 25 Jun 2024.

Builds on

31 of 49 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  8. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  9. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  10. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  11. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  12. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
19 more
  1. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  2. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  3. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  4. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  5. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  6. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
  7. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  8. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 374
  9. 2023In-Context Learning Creates Task Vectorscited by 302
  10. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  11. 2018Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfercited by 209
  12. 2023Linear Representations of Sentiment in Large Language Modelscited by 157
  13. 2021XFORMAL: A Benchmark for Multilingual Formality Style Transfercited by 51
  14. 2022Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Modelscited by 49
  15. 2023Self-Detoxifying Language Models via Toxification Reversalcited by 45
  16. 2022As Little as Possible, as Much as Necessary: Detecting Over- and Undertranslations with Contrastive Conditioningcited by 11
  17. 2022Detecting Text Formality: A Study of Text Classification Approachescited by 11
  18. 2023RAMP: Retrieval and Attribute-Marking Enhanced Prompting for Attribute-Controlled Translationcited by 9
  19. 2020ETC-NLG: End-to-end Topic-Conditioned Natural Language Generationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 11; 2026: 20.

35 papers.

Sort:

By area:

  1. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  2. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 149
  3. 2025To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Modelscited by 26
  4. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 23
  5. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 18
  6. 2025The Rogue Scalpel: Activation Steering Compromises LLM Safetycited by 18
  7. 2025Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Controlcited by 14
  8. 2024Gumbel Counterfactual Generation From Language Modelscited by 13
  9. 2026Steering Vector Fields for Context-Aware Inference-Time Control in Large Language Modelscited by 10
  10. 2026Why Steering Works: Toward a Unified View of Language Model Parameter Dynamicscited by 8
  11. 2026Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Controlcited by 8
  12. 2025Designing Role Vectors to Improve LLM Inference Behaviourcited by 5
23 more
  1. 2025Refusal Steering: Fine-grained Control over LLM Refusal Behaviour for Sensitive Topicscited by 5
  2. 2026Compositional Steering of Large Language Models with Steering Tokenscited by 5
  3. 2025A Comparative Analysis of Sparse Autoencoder and Activation Difference in Language Model Steeringcited by 3
  4. 2025Steering Large Language Models for Machine Translation Personalizationcited by 2
  5. 2025Steering Information Utility in Key-Value Memory for Language Model Post-Trainingcited by 2
  6. 2026Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Controlcited by 2
  7. 2026Neural Procedural Memory: Empowering LLM Agents with Implicit Activation Steeringcited by 2
  8. 2026Deployable Per-Instance Multi-Layer Activation Steering for Large Language Modelscited by 2
  9. 2024A gentle push funziona benissimo: making instructed models in Italian via contrastive activation steeringcited by 1
  10. 2025DAPI: Domain Adaptive Toxicity Probe Vector Intervention for Fine-Grained Detoxificationcited by 1
  11. 2026Funny or Persuasive, but Not Both: Evaluating Fine-Grained Multi-Concept Control in LLMscited by 1
  12. 2026Depth Charge: Jailbreak Large Language Models from Deep Safety Attention Headscited by 1
  13. 2026AI Steerability 360: A Toolkit for Steering Large Language Modelscited by 1
  14. 2026Knowledge Vector of Logical Reasoning in Large Language Modelscited by 0
  15. 2026Activation Steering for Synthetic Data Generation: The Role of Diversity in Downstream Safety Detectioncited by 0
  16. 2026Policy Gradient Steering: Interventions from Behavioral Objectivescited by 0
  17. 2026Inference-Time Policy Alignment for Fair Reinforcement Learningcited by 0
  18. 2026SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalizationcited by 0
  19. 2026Can Activation Steering Capture Multidimensional Authorship Style?cited by 0
  20. 2026Metacognitive Steering: Learning the Structure of Scientific Judgmentcited by 0
  21. 2026Sampling Reveals Style: Unsupervised, Training-Free Discovery of Prompt-Conditional Stylistic Axes in LLM Activationscited by 0
  22. 2026Adaptive Multi-Value Control in LLMs via Causal Activation Steeringcited by 0
  23. 2026How to Tame a Multi-Headed Hydra? Adaptive Multi-Category Safety Steering for Large Language Modelscited by 0