# How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularities

arXiv: https://arxiv.org/abs/2603.02578 · HTML: https://mlsanity.com/paper/2603.02578/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 130 references that are in the map.
- First posted on arXiv 3 Mar 2026.

## Builds on (most cited first, 30 of 30)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators](https://mlsanity.com/paper/2404.04475/) (2024)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [Persona Vectors: Monitoring and Controlling Character Traits in Language Models](https://mlsanity.com/paper/2507.21509/) (2025)
- [AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders](https://mlsanity.com/paper/2501.17148/) (2025)
- [Open Problems in Mechanistic Interpretability](https://mlsanity.com/paper/2501.16496/) (2025)
- [Linear Representations of Sentiment in Large Language Models](https://mlsanity.com/paper/2310.15154/) (2023)
- [Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization](https://mlsanity.com/paper/2406.00045/) (2024)
- [A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models](https://mlsanity.com/paper/2503.05613/) (2025)
- [Steering Large Language Model Activations in Sparse Spaces](https://mlsanity.com/paper/2503.00177/) (2025)
- [AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraint](https://mlsanity.com/paper/2506.07022/) (2025)
- [A Unified Understanding and Evaluation of Steering Methods](https://mlsanity.com/paper/2502.02716/) (2025)
- [Towards Reliable Evaluation of Behavior Steering Interventions in LLMs](https://mlsanity.com/paper/2410.17245/) (2024)
- [Personalized Text Generation with Contrastive Activation Steering](https://mlsanity.com/paper/2503.05213/) (2025)
- [Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering](https://mlsanity.com/paper/2511.00617/) (2025)
- [Mechanistic Interpretability of Emotion Inference in Large Language Models](https://mlsanity.com/paper/2502.05489/) (2025)
- [Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atoms](https://mlsanity.com/paper/2505.20322/) (2025)
- [HyperSteer: Activation Steering at Scale with Hypernetworks](https://mlsanity.com/paper/2506.03292/) (2025)
- [Representation Engineering for Large-Language Models: Survey and Research Challenges](https://mlsanity.com/paper/2502.17601/) (2025)
- [Improved Representation Steering for Language Models](https://mlsanity.com/paper/2505.20809/) (2025)
- [ChatGPT is a Remarkable Tool -- For Experts](https://mlsanity.com/paper/2306.03102/) (2023)
- [The Shape of Beliefs: Geometry, Dynamics, and Interventions along Representation Manifolds of Language Models' Posteriors](https://mlsanity.com/paper/2602.02315/) (2026)
- [The Information Geometry of Softmax: Probing and Steering](https://mlsanity.com/paper/2602.15293/) (2026)
- [Why Steering Works: Toward a Unified View of Language Model Parameter Dynamics](https://mlsanity.com/paper/2602.02343/) (2026)

## Built on by (most cited first, 3 of 3)

- [What Does Activation Steering Control? Attribution Across Answer Encodings and Output-Sensitive Subspaces](https://mlsanity.com/paper/2608.22985/) (2026)
- [RepBench: Compiling Benchmarks into Capability Representations for Large Language Models](https://mlsanity.com/paper/2607.28008/) (2026)
- [When Do Model Internals Help? Exploring the Role of Representation Engineering in LLM Safety](https://mlsanity.com/paper/2609.34771/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly deployed in socially sensitive domains, yet their unpredictable behaviors, ranging from misaligned intent to inconsistent personality, pose significant risks. We introduce SteerEval, a hierarchical benchmark for evaluating LLM controllability across three domains: language features, sentiment, and personality. Each domain is structured into three specification levels: L1 (what to express), L2 (how to express), and L3 (how to instantiate), connecting high-level behavioral intent to concrete textual output. Using SteerEval, we systematically evaluate contemporary steering methods, revealing that control often degrades at finer-grained levels. Our benchmark offers a principled and interpretable framework for safe and controllable LLM behavior, serving as a foundation for future research.