# Context Steering: Controllable Personalization at Inference Time

arXiv: https://arxiv.org/abs/2405.01768 · HTML: https://mlsanity.com/paper/2405.01768/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2024.

## Built on by (most cited first, 22 of 22)

- [Think-While-Generating: On-the-Fly Reasoning for Personalized Long-Form Generation](https://mlsanity.com/paper/2512.06690/) (2025)
- [Spotlight Your Instructions: Instruction-following with Dynamic Attention Steering](https://mlsanity.com/paper/2505.12025/) (2025)
- [Implicit Personalization in Language Models: A Systematic Study](https://mlsanity.com/paper/2405.14808/) (2024)
- [CoSteer: Collaborative Decoding-Time Personalization via Local Delta Steering](https://mlsanity.com/paper/2507.04756/) (2025)
- [CoCoEmo: Composable and Controllable Human-Like Emotional TTS via Activation Steering](https://mlsanity.com/paper/2602.03420/) (2026)
- [COLD-Steer: Steering Large Language Models via In-Context One-step Learning Dynamics](https://mlsanity.com/paper/2603.06495/) (2026)
- [SteerX: Disentangled Steering for LLM Personalization](https://mlsanity.com/paper/2510.22256/) (2025)
- [Your Language Model Secretly Contains Personality Subnetworks](https://mlsanity.com/paper/2602.07164/) (2026)
- [From Words to Widgets for Controllable LLM Generation](https://mlsanity.com/paper/2604.10925/) (2026)
- [Rethinking Prompt-based Debiasing in Large Language Models](https://mlsanity.com/paper/2503.09219/) (2025)
- [EXACT: Explicit Attribute-Guided Decoding-Time Personalization](https://mlsanity.com/paper/2602.17695/) (2026)
- [Self-Improving Model Steering](https://mlsanity.com/paper/2507.08967/) (2025)
- [Communicate to Play: Pragmatic Reasoning for Efficient Cross-Cultural Communication in Codenames](https://mlsanity.com/paper/2408.04900/) (2024)
- [SDA: Steering-Driven Distribution Alignment for Open LLMs without Fine-Tuning](https://mlsanity.com/paper/2511.16324/) (2025)
- [Steerable Chatbots: Exploring Personalization Control Interfaces via LLM Activation Steering](https://mlsanity.com/paper/2505.04260/) (2025)
- [URLs Help, Topics Guide: Understanding Metadata Utility in LLM Training](https://mlsanity.com/paper/2505.16570/) (2025)
- [ShallowJail: Steering Jailbreaks against Large Language Models](https://mlsanity.com/paper/2602.07107/) (2026)
- [PrefReward: Learning User Preference Matrix for Personalized Text Generation](https://mlsanity.com/paper/2607.21067/) (2026)
- [Interpolative Decoding: Exploring the Spectrum of Personality Traits in LLMs](https://mlsanity.com/paper/2512.19937/) (2025)
- [MATO: Multi-objective Personalized Alignment with Test-time Optimization for Large Language Models](https://mlsanity.com/paper/2605.25342/) (2026)
- [Large Language Models Should Learn Personalized Rather Than Aggregated Human Preferences](https://mlsanity.com/paper/2606.07629/) (2026)
- [Cautious Context Steering for Language Model Personalization](https://mlsanity.com/paper/2608.05813/) (2026)

## Abstract (arXiv)

To deliver high-quality, personalized responses, large language models (LLMs) must effectively incorporate context -- personal, demographic, and cultural information specific to an end-user. For example, asking the model to explain Newton's second law with the context "I am a toddler" should produce a response different from when the context is "I am a physics professor". However, leveraging the context in practice is a nuanced and challenging task, and is often dependent on the specific situation or user base. The model must strike a balance between providing specific, personalized responses and maintaining general applicability. Current solutions, such as prompt-engineering and fine-tuning, require collection of contextually appropriate responses as examples, making them time-consuming and less flexible to use across different contexts. In this work, we introduce Context Steering (CoS) -- a simple, training-free decoding approach that amplifies the influence of the context in next token predictions. CoS computes contextual influence by comparing the output probabilities from two LLM forward passes: one that includes the context and one that does not. By linearly scaling the contextual influence, CoS allows practitioners to flexibly control the degree of personalization for different use cases. We show that CoS can be applied to autoregressive LLMs, and demonstrates strong performance in personalized recommendations. Additionally, we show that CoS can function as a Bayesian Generative model to infer and quantify correlations between open-ended texts, broadening its potential applications.