# Personalized Text Generation with Contrastive Activation Steering

arXiv: https://arxiv.org/abs/2503.05213 · HTML: https://mlsanity.com/paper/2503.05213/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability.
- Builds on 28 of its 37 references that are in the map.
- First posted on arXiv 7 Mar 2025.

## Placed in

- Mechanistic interpretability / [Activation steering methods](https://mlsanity.com/mechinterp/activation-steering-methods/)

## Builds on (most cited first, 28 of 28)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging](https://mlsanity.com/paper/2310.11564/) (2023)
- [In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering](https://mlsanity.com/paper/2311.06668/) (2023)
- [Personalization of Large Language Models: A Survey](https://mlsanity.com/paper/2411.00027/) (2024)
- [Optimization Methods for Personalizing Large Language Models through Retrieval Augmentation](https://mlsanity.com/paper/2404.05970/) (2024)
- [LongLaMP: A Benchmark for Personalized Long-form Text Generation](https://mlsanity.com/paper/2407.11016/) (2024)
- [TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space](https://mlsanity.com/paper/2402.17811/) (2024)
- [Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning](https://mlsanity.com/paper/2402.04401/) (2024)
- [Personalized Pieces: Efficient Personalized Large Language Models through Collaborative Efforts](https://mlsanity.com/paper/2406.10471/) (2024)
- [Large Language Models Empowered Personalized Web Agents](https://mlsanity.com/paper/2410.17236/) (2024)
- [HYDRA: Model Factorization Framework for Black-Box LLM Personalization](https://mlsanity.com/paper/2406.02888/) (2024)
- [Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning](https://mlsanity.com/paper/2312.17484/) (2023)
- [Unsupervised Dense Retrieval with Relevance-Aware Contrastive Pre-Training](https://mlsanity.com/paper/2306.03166/) (2023)
- [Personalized Text Generation with Fine-Grained Linguistic Control](https://mlsanity.com/paper/2402.04914/) (2024)
- [StyleRemix: Interpretable Authorship Obfuscation via Distillation and Perturbation of Style Elements](https://mlsanity.com/paper/2408.15666/) (2024)
- [Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximization](https://mlsanity.com/paper/2410.09942/) (2024)
- [Can Large Language Models Understand Preferences in Personalized Recommendation?](https://mlsanity.com/paper/2501.13391/) (2025)
- [Using Prompts to Guide Large Language Models in Imitating a Real Person's Language Style](https://mlsanity.com/paper/2410.03848/) (2024)
- [Investigating Large Language Models in Inferring Personality Traits from User Conversations](https://mlsanity.com/paper/2501.07532/) (2025)
- [Personalized Language Model Learning on Text Data Without User Identifiers](https://mlsanity.com/paper/2501.06062/) (2025)

## Built on by (most cited first, 30 of 34)

- [Latent Inter-User Difference Modeling for LLM Personalization](https://mlsanity.com/paper/2507.20849/) (2025)
- [BILLY: Steering Large Language Models via Merging Persona Vectors for Creative Generation](https://mlsanity.com/paper/2510.10157/) (2025)
- [SteerX: Disentangled Steering for LLM Personalization](https://mlsanity.com/paper/2510.22256/) (2025)
- [CURP: Codebook-based Continuous User Representation for Personalized Generation with LLMs](https://mlsanity.com/paper/2602.00742/) (2026)
- [Evaluating Style-Personalized Text Generation: Challenges and Directions](https://mlsanity.com/paper/2508.06374/) (2025)
- [GRAVITY: A Framework for Personalized Text Generation via Profile-Grounded Synthetic Preferences](https://mlsanity.com/paper/2510.11952/) (2025)
- [Text as a Universal Interface for Transferable Personalization](https://mlsanity.com/paper/2601.04963/) (2026)
- [CARD: Cluster-level Adaptation with Reward-guided Decoding for Personalized Text Generation](https://mlsanity.com/paper/2601.06352/) (2026)
- [How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularities](https://mlsanity.com/paper/2603.02578/) (2026)
- [HotComment: A Benchmark for Evaluating Popularity of Online Comments](https://mlsanity.com/paper/2604.25614/) (2026)
- [Knowing without Acting: The Disentangled Geometry of Safety Mechanisms in Large Language Models](https://mlsanity.com/paper/2603.05773/) (2026)
- [Preference Heads in Large Language Models: A Mechanistic Framework for Interpretable Personalization](https://mlsanity.com/paper/2604.22345/) (2026)
- [Steerable Chatbots: Exploring Personalization Control Interfaces via LLM Activation Steering](https://mlsanity.com/paper/2505.04260/) (2025)
- [Disco-RAG: Discourse-Aware Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.04377/) (2026)
- [A Concise Agent is Less Expert: Revealing Side Effects of Using Style Features on Conversational Agents](https://mlsanity.com/paper/2601.10809/) (2026)
- [Uncertainty-Aware Variational Reward Factorization via Probabilistic Preference Bases for LLM Personalization](https://mlsanity.com/paper/2604.00997/) (2026)
- [CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors](https://mlsanity.com/paper/2604.14773/) (2026)
- [Beyond Isolated Behaviors: Hierarchical User Modeling for LLM Personalization](https://mlsanity.com/paper/2606.02300/) (2026)
- [Self-supervised User Profile Generation for Personalization](https://mlsanity.com/paper/2606.05336/) (2026)
- [PrefReward: Learning User Preference Matrix for Personalized Text Generation](https://mlsanity.com/paper/2607.21067/) (2026)
- [Do Implicit Personalization and Explicit Styles Conflict? PsPLUG: A Lightweight Plug-in for Balancing Personalization and Style in Customized LLMs](https://mlsanity.com/paper/2601.06362/) (2026)
- [Mechanistic Indicators of Steering Effectiveness in Large Language Models](https://mlsanity.com/paper/2602.01716/) (2026)
- [The Subjectivity of Respect in Police Traffic Stops: Modeling Community Perspectives in Body-Worn Camera Footage](https://mlsanity.com/paper/2602.10339/) (2026)
- [Steered Generation via Gradient-Based Optimization on Sparse Query Features](https://mlsanity.com/paper/2605.23040/) (2026)
- [From Profiles to Steering Vectors: Global Sparse Priors and Local Semantic Calibration for Personalized Text Generation](https://mlsanity.com/paper/2607.21620/) (2026)
- [Learning Preference Adaptation for Large Language Model Personalization via Verbal Reinforcement Learning](https://mlsanity.com/paper/2608.09507/) (2026)
- [Knowledge-Graph-Gated Defactualization for Style-Controllable and Fact-Preserving Generation in Agentic Conversational AI](https://mlsanity.com/paper/2608.20393/) (2026)
- [LITERARYBIGFIVE: Author-Personalized Text Generation in a Unified Interpretable Space](https://mlsanity.com/paper/2608.23124/) (2026)
- [Controlling and Assessing Appropriate Persona Use in LLM-based Dialogue Generation](https://mlsanity.com/paper/2609.04676/) (2026)
- [Can Activation Steering Capture Multidimensional Authorship Style?](https://mlsanity.com/paper/2609.04792/) (2026)

## Abstract (arXiv)

Personalized text generation aims to infer users' writing style preferences from their historical texts and generate outputs that faithfully reflect these stylistic characteristics. Existing solutions primarily adopt two paradigms: retrieval-augmented generation (RAG) and parameter-efficient fine-tuning (PEFT). While these approaches have advanced the field, they suffer from two critical limitations: (1) the entanglement of content semantics and stylistic patterns in historical texts impedes accurate modeling of user-specific writing preferences; and (2) scalability challenges arising from both RAG's inference latency by retrieval operations and PEFT's parameter storage requirements for per user model. To overcome these limitations, we propose StyleVector, a training-free framework that disentangles and represents personalized writing style as a vector in LLM's activation space, enabling style-steered generation during inference without requiring costly retrieval or parameter storage. Comprehensive experiments demonstrate that our framework achieves a significant 8% relative improvement in personalized generation while reducing storage requirements by 1700 times over PEFT method.