# Compositional Steering of Large Language Models with Steering Tokens

arXiv: https://arxiv.org/abs/2601.05062 · HTML: https://mlsanity.com/paper/2601.05062/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 39 references that are in the map.
- First posted on arXiv 8 Jan 2026.

## Builds on (most cited first, 30 of 32)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch](https://mlsanity.com/paper/2311.03099/) (2023)
- [Learning to Compress Prompts with Gist Tokens](https://mlsanity.com/paper/2304.08467/) (2023)
- [Fine-tuning Language Models for Factuality](https://mlsanity.com/paper/2311.08401/) (2023)
- [Improving Instruction-Following in Language Models through Activation Steering](https://mlsanity.com/paper/2410.12877/) (2024)
- [Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization](https://mlsanity.com/paper/2406.00045/) (2024)
- [What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineering](https://mlsanity.com/paper/2406.12334/) (2024)
- [Word Embeddings Are Steers for Language Models](https://mlsanity.com/paper/2305.12798/) (2023)
- [Prompt Compression for Large Language Models: A Survey](https://mlsanity.com/paper/2410.12388/) (2024)
- [Steering Large Language Model Activations in Sparse Spaces](https://mlsanity.com/paper/2503.00177/) (2025)
- [A Unified Understanding and Evaluation of Steering Methods](https://mlsanity.com/paper/2502.02716/) (2025)
- [Multi-Attribute Steering of Language Models via Targeted Intervention](https://mlsanity.com/paper/2502.12446/) (2025)
- [Extending Activation Steering to Broad Skills and Multiple Behaviours](https://mlsanity.com/paper/2403.05767/) (2024)
- [Multi-property Steering of Large Language Models with Dynamic Activation Composition](https://mlsanity.com/paper/2406.17563/) (2024)
- [Steering off Course: Reliability Challenges in Steering Language Models](https://mlsanity.com/paper/2504.04635/) (2025)
- [Multimodal Distillation for Egocentric Action Recognition](https://mlsanity.com/paper/2307.07483/) (2023)
- [Strong and weak alignment of large language models with human values](https://mlsanity.com/paper/2408.04655/) (2024)
- [Towards LLMs Robustness to Changes in Prompt Format Styles](https://mlsanity.com/paper/2504.06969/) (2025)
- [Improved Representation Steering for Language Models](https://mlsanity.com/paper/2505.20809/) (2025)
- [How Many Instructions Can LLMs Follow at Once?](https://mlsanity.com/paper/2507.11538/) (2025)
- [The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge](https://mlsanity.com/paper/2509.26072/) (2025)
- [Long Context In-Context Compression by Getting to the Gist of Gisting](https://mlsanity.com/paper/2504.08934/) (2025)
- [Say More with Less: Understanding Prompt Learning Behaviors through Gist Compression](https://mlsanity.com/paper/2402.16058/) (2024)
- [GistScore: Learning Better Representations for In-Context Example Selection with Gist Bottlenecks](https://mlsanity.com/paper/2311.09606/) (2023)
- [ReCoVeR the Target Language: Language Steering without Sacrificing Task Performance](https://mlsanity.com/paper/2509.14814/) (2025)
- [Investigating the Effectiveness of HyperTuning via Gisting](https://mlsanity.com/paper/2402.16817/) (2024)
- [Sentence-Anchored Gist Compression for Long-Context LLMs](https://mlsanity.com/paper/2511.08128/) (2025)

## Built on by (most cited first, 5 of 5)

- [Skill Neologisms: Towards Skill-based Continual Learning](https://mlsanity.com/paper/2605.04970/) (2026)
- [ORBIT: Training-Free Multi-Attribute Behavioral Steering via Orthogonal Subspace Rotation](https://mlsanity.com/paper/2606.22357/) (2026)
- [Spillover-Aware Multi-Value Steering for Pluralistic LLM Alignment](https://mlsanity.com/paper/2609.05800/) (2026)
- [Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation](https://mlsanity.com/paper/2606.18389/) (2026)
- [Policy Gradient Steering: Interventions from Behavioral Objectives](https://mlsanity.com/paper/2607.27574/) (2026)

## Abstract (arXiv)

Deploying LLMs in real-world applications requires controllable output that satisfies multiple desiderata at the same time. While existing work extensively addresses LLM steering for a single behavior, textitcompositional steering -- i.e., steering LLMs simultaneously towards multiple behaviors -- remains an underexplored problem. In this work, we propose compositional steering tokens for multi-behavior steering. We first embed individual behaviors, expressed as natural language instructions, into dedicated tokens via self-distillation. Contrary to most prior work, which operates in the activation space, our behavior steers live in the space of input tokens, enabling more effective zero-shot composition. We then train a dedicated textitcomposition token on pairs of behaviors and show that it successfully captures the notion of composition: it generalizes well to textitunseen compositions, including those with unseen behaviors as well as those with an unseen textitnumber of behaviors. Our experiments across different LLM architectures show that steering tokens lead to superior multi-behavior steering of verifiable constraints (e.g., length, format, structure, language) compared to competing approaches (instructions, activation steering, and LoRA merging). Moreover, we show that steering tokens complement natural language instructions, with their combination resulting in further gains.