Controlling Chat Style in Language Models via Single-Direction Editing
Not on the map. Why?
Abstract (arXiv)
Controlling stylistic attributes in large language models (LLMs) remains challenging, with existing approaches relying on either prompt engineering or post-training alignment. This paper investigates this challenge through the lens of representation engineering, testing the hypothesis that distinct stylistic attributes - from emotional tone to linguistic structure - are encoded as linear directions in the model's activation space. We provide strong empirical evidence for this hypothesis across a wide range of styles and, based on this finding, present a lightweight, training-free method for precise style control. Our approach supports linear style composition, enhances safety by ablating undesirable behaviors, and, as confirmed by experiments on over a dozen models, achieves high style adherence while preserving core capabilities at minimal computational cost.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 14 of its 16 references that are in the map. First posted on arXiv 10 Feb 2026.
Builds on
14 of 16 references.
References · 14
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
- 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
- 2021Knowledge Neurons in Pretrained Transformerscited by 590
- 2023LEACE: Perfect linear concept erasure in closed formcited by 254
Built on by
Not cited yet.