# Style Vectors for Steering Generative Large Language Model

arXiv: https://arxiv.org/abs/2402.01618 · HTML: https://mlsanity.com/paper/2402.01618/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 101 references that are in the map.
- First posted on arXiv 2 Feb 2024.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models](https://mlsanity.com/paper/2211.05105/) (2022)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts](https://mlsanity.com/paper/2110.01691/) (2021)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do](https://mlsanity.com/paper/2103.11790/) (2021)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [Shakespearizing Modern Language Using Copy-Enriched Sequence-to-Sequence Models](https://mlsanity.com/paper/1707.01161/) (2017)
- [Hooks in the Headline: Learning to Generate Headlines with Controlled Styles](https://mlsanity.com/paper/2004.01980/) (2020)
- [Structuring Latent Spaces for Stylized Response Generation](https://mlsanity.com/paper/1909.05361/) (2019)
- [The Stable Artist: Steering Semantics in Diffusion Latent Space](https://mlsanity.com/paper/2212.06013/) (2022)
- [StyleDGPT: Stylized Response Generation with Pre-trained Language Models](https://mlsanity.com/paper/2010.02569/) (2020)
- [Stylized Knowledge-Grounded Dialogue Generation via Disentangled Template Rewriting](https://mlsanity.com/paper/2204.05610/) (2022)

## Built on by (most cited first, 30 of 56)

- [Programming Refusal with Conditional Activation Steering](https://mlsanity.com/paper/2409.05907/) (2024)
- [MemOS: A Memory OS for AI System](https://mlsanity.com/paper/2507.03724/) (2025)
- [Angular Steering: Behavior Control via Rotation in Activation Space](https://mlsanity.com/paper/2510.26243/) (2025)
- [Understanding (Un)Reliability of Steering Vectors in Language Models](https://mlsanity.com/paper/2505.22637/) (2025)
- [Beyond Single Concept Vector: Modeling Concept Subspace in LLMs with Gaussian Distribution](https://mlsanity.com/paper/2410.00153/) (2024)
- [Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atoms](https://mlsanity.com/paper/2505.20322/) (2025)
- [The Rogue Scalpel: Activation Steering Compromises LLM Safety](https://mlsanity.com/paper/2509.22067/) (2025)
- [LiSeCo: Linear Semantic Control for Language Generation](https://mlsanity.com/paper/2405.15454/) (2024)
- [ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Models](https://mlsanity.com/paper/2503.22048/) (2025)
- [Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Control](https://mlsanity.com/paper/2604.03147/) (2026)
- [Differentially Private Steering for Large Language Model Alignment](https://mlsanity.com/paper/2501.18532/) (2025)
- [Activation Steering with a Feedback Controller](https://mlsanity.com/paper/2510.04309/) (2025)
- [Multi-objective Large Language Model Alignment with Hierarchical Experts](https://mlsanity.com/paper/2505.20925/) (2025)
- [Words in Motion: Extracting Interpretable Control Vectors for Motion Transformers](https://mlsanity.com/paper/2406.11624/) (2024)
- [Activation Steering for Masked Diffusion Language Models](https://mlsanity.com/paper/2512.24143/) (2025)
- [Why Steering Works: Toward a Unified View of Language Model Parameter Dynamics](https://mlsanity.com/paper/2602.02343/) (2026)
- [Mitigating Hallucination in VideoLLMs via Temporal-Aware Activation Engineering](https://mlsanity.com/paper/2505.12826/) (2025)
- [Efficient and accurate steering of Large Language Models through attention-guided feature learning](https://mlsanity.com/paper/2602.00333/) (2026)
- [Effectively Steer LLM To Follow Preference via Building Confident Directions](https://mlsanity.com/paper/2503.02989/) (2025)
- [EmoShift: Lightweight Activation Steering for Enhanced Emotion-Aware Speech Synthesis](https://mlsanity.com/paper/2601.22873/) (2026)
- [Fine-Grained Activation Steering: Steering Less, Achieving More](https://mlsanity.com/paper/2602.04428/) (2026)
- [Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs](https://mlsanity.com/paper/2602.07276/) (2026)
- [What's Producible May Not Be Reachable: Measuring the Steerability of Generative Models](https://mlsanity.com/paper/2503.17482/) (2025)
- [Silence is Golden: Mitigating Hallucinations in Large Audio-Language Models via Layer-Weighted Vector Steering](https://mlsanity.com/paper/2510.12851/) (2025)
- [The Effectiveness of Style Vectors for Steering Large Language Models: A Human Evaluation](https://mlsanity.com/paper/2601.21505/) (2026)
- [Bottlenecked Transformers: Periodic KV Cache Consolidation for Generalised Reasoning](https://mlsanity.com/paper/2505.16950/) (2025)
- [We Think, Therefore We Align LLMs to Helpful, Harmless and Honest Before They Go Wrong](https://mlsanity.com/paper/2509.22510/) (2025)
- [Machine Unlearning Meets Adversarial Robustness via Constrained Interventions on LLMs](https://mlsanity.com/paper/2510.03567/) (2025)
- [PersRM-R1: Enhance Personalized Reward Modeling with Reinforcement Learning](https://mlsanity.com/paper/2508.14076/) (2025)
- [Control Reinforcement Learning: Interpretable Token-Level Steering of LLMs via Sparse Autoencoder Features](https://mlsanity.com/paper/2602.10437/) (2026)

## Abstract (arXiv)

This research explores strategies for steering the output of large language models (LLMs) towards specific styles, such as sentiment, emotion, or writing style, by adding style vectors to the activations of hidden layers during text generation. We show that style vectors can be simply computed from recorded layer activations for input texts in a specific style in contrast to more complex training-based approaches. Through a series of experiments, we demonstrate the effectiveness of activation engineering using such style vectors to influence the style of generated text in a nuanced and parameterisable way, distinguishing it from prompt engineering. The presented research constitutes a significant step towards developing more adaptive and effective AI-empowered interactive systems.