# Evaluating the Prompt Steerability of Large Language Models

arXiv: https://arxiv.org/abs/2411.12405 · HTML: https://mlsanity.com/paper/2411.12405/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 47 references that are in the map.
- First posted on arXiv 19 Nov 2024.

## Builds on (most cited first, 30 of 34)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Larger language models do in-context learning differently](https://mlsanity.com/paper/2303.03846/) (2023)
- [Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models](https://mlsanity.com/paper/2305.18189/) (2023)
- [Programming Refusal with Conditional Activation Steering](https://mlsanity.com/paper/2409.05907/) (2024)
- [A Roadmap to Pluralistic Alignment](https://mlsanity.com/paper/2402.05070/) (2024)
- [Fundamental Limitations of Alignment in Large Language Models](https://mlsanity.com/paper/2304.11082/) (2023)
- [The Learnability of In-Context Learning](https://mlsanity.com/paper/2303.07895/) (2023)
- [Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration](https://mlsanity.com/paper/2406.15951/) (2024)
- [Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties](https://mlsanity.com/paper/2309.00779/) (2023)
- [PersonaGym: Evaluating Persona Agents and LLMs](https://mlsanity.com/paper/2407.18416/) (2024)
- [Guiding Large Language Models via Directional Stimulus Prompting](https://mlsanity.com/paper/2302.11520/) (2023)
- [Evaluating Large Language Model Biases in Persona-Steered Generation](https://mlsanity.com/paper/2405.20253/) (2024)
- [Word Embeddings Are Steers for Language Models](https://mlsanity.com/paper/2305.12798/) (2023)
- ["My Answer is C": First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models](https://mlsanity.com/paper/2402.14499/) (2024)
- [Aligning LLMs with Individual Preferences via Interaction](https://mlsanity.com/paper/2410.03642/) (2024)
- [When Large Language Models contradict humans? Large Language Models' Sycophantic Behaviour](https://mlsanity.com/paper/2311.09410/) (2023)
- [DeAL: Decoding-time Alignment for Large Language Models](https://mlsanity.com/paper/2402.06147/) (2024)
- [On the steerability of large language models toward data-driven personas](https://mlsanity.com/paper/2311.04978/) (2023)
- [Steering Large Language Models for Machine Translation with Finetuning and In-Context Learning](https://mlsanity.com/paper/2310.13448/) (2023)
- [Steering Without Side Effects: Improving Post-Deployment Control of Language Models](https://mlsanity.com/paper/2406.15518/) (2024)
- [What's the Magic Word? A Control Theory of LLM Prompting](https://mlsanity.com/paper/2310.04444/) (2023)
- [How Do Nonlinear Transformers Learn and Generalize in In-Context Learning?](https://mlsanity.com/paper/2402.15607/) (2024)
- [What are human values, and how do we align AI to them?](https://mlsanity.com/paper/2404.10636/) (2024)
- [Language Models in Dialogue: Conversational Maxims for Human-AI Interactions](https://mlsanity.com/paper/2403.15115/) (2024)

## Built on by (most cited first, 27 of 27)

- [AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agents](https://mlsanity.com/paper/2506.04018/) (2025)
- [A Course Correction in Steerability Evaluation: Revealing Miscalibration and Side Effects in LLMs](https://mlsanity.com/paper/2505.23816/) (2025)
- [What's Producible May Not Be Reachable: Measuring the Steerability of Generative Models](https://mlsanity.com/paper/2503.17482/) (2025)
- [Poor Alignment and Steerability of Large Language Models: Evidence from College Admission Essays](https://mlsanity.com/paper/2503.20062/) (2025)
- [What Can We Actually Steer? A Multi-Behavior Study of Activation Control](https://mlsanity.com/paper/2511.18284/) (2025)
- [Quantifying the Impact of Structured Output Format on Large Language Models through Causal Inference](https://mlsanity.com/paper/2509.21791/) (2025)
- [Security Steerability is All You Need](https://mlsanity.com/paper/2504.19521/) (2025)
- [Localizing Persona Representations in LLMs](https://mlsanity.com/paper/2505.24539/) (2025)
- [An Auditable Agent Platform For Automated Molecular Optimisation](https://mlsanity.com/paper/2508.03444/) (2025)
- [AI Steerability 360: A Toolkit for Steering Large Language Models](https://mlsanity.com/paper/2603.07837/) (2026)
- [FaithSteer-BENCH: A Deployment-Aligned Stress-Testing Benchmark for Inference-Time Steering](https://mlsanity.com/paper/2603.18329/) (2026)
- [What Does the AI Doctor Value? Auditing Pluralism in the Clinical Ethics of Language Models](https://mlsanity.com/paper/2605.18738/) (2026)
- [Inference-Time Intervention in Large Language Models for Reliable Requirement Verification](https://mlsanity.com/paper/2503.14130/) (2025)
- [Music Recommendation with Large Language Models: Challenges, Opportunities, and Evaluation](https://mlsanity.com/paper/2511.16478/) (2025)
- [Left, Right, or Center? Evaluating LLM Framing in News Classification and Generation](https://mlsanity.com/paper/2601.05835/) (2026)
- [Predicting Biased Human Decision-Making with Large Language Models in Conversational Settings](https://mlsanity.com/paper/2601.11049/) (2026)
- [Emulating Aggregate Human Choice Behavior and Biases with GPT Conversational Agents](https://mlsanity.com/paper/2602.05597/) (2026)
- [Not All Subjectivity Is the Same! Defining Desiderata for the Evaluation of Subjectivity in NLP](https://mlsanity.com/paper/2603.28351/) (2026)
- [Toward Stable Value Alignment: Introducing Independent Modules for Consistent Value Guidance](https://mlsanity.com/paper/2605.11712/) (2026)
- [How Far Will They Go? Red-Teaming Online Influence with Large Language Models](https://mlsanity.com/paper/2605.22880/) (2026)
- [On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance](https://mlsanity.com/paper/2606.00467/) (2026)
- [Investigating Linguistic Steering: An Analysis of Adjectival Effects Across Large Language Model Architectures](https://mlsanity.com/paper/2606.20572/) (2026)
- [Auditing Alignment Controllability in LLMs via Political Axes](https://mlsanity.com/paper/2607.23519/) (2026)
- [Controllable Affective Generation via Latent Vector Steering](https://mlsanity.com/paper/2608.25569/) (2026)
- [CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systems](https://mlsanity.com/paper/2609.22494/) (2026)
- [How Far Do Persona Effects Generalize in Language Models?](https://mlsanity.com/paper/2609.32758/) (2026)
- [MetaSteer: Context-Conditioned, nonlinear Steering via Attention-Projection Adaptation](https://mlsanity.com/paper/2609.38718/) (2026)

## Abstract (arXiv)

Building pluralistic AI requires designing models that are able to be shaped to represent a wide range of value systems and cultures. Achieving this requires first being able to evaluate the degree to which a given model is capable of reflecting various personas. To this end, we propose a benchmark for evaluating the steerability of model personas as a function of prompting. Our design is based on a formal definition of prompt steerability, which analyzes the degree to which a model's joint behavioral distribution can be shifted from its baseline. By defining steerability indices and inspecting how these indices change as a function of steering effort, we can estimate the steerability of a model across various persona dimensions and directions. Our benchmark reveals that the steerability of many current models is limited -- due to both a skew in their baseline behavior and an asymmetry in their steerability across many persona dimensions. We release an implementation of our benchmark at https://github.com/IBM/prompt-steering.