mlsanityData to 14 Oct 2026

Evaluating the Prompt Steerability of Large Language Models

arXiv 2411.12405 ↗First posted 19 Nov 2024cs.CL cs.AI cs.HCcited by 27

Not on the map. Why?

Abstract (arXiv)

Building pluralistic AI requires designing models that are able to be shaped to represent a wide range of value systems and cultures. Achieving this requires first being able to evaluate the degree to which a given model is capable of reflecting various personas. To this end, we propose a benchmark for evaluating the steerability of model personas as a function of prompting. Our design is based on a formal definition of prompt steerability, which analyzes the degree to which a model's joint behavioral distribution can be shifted from its baseline. By defining steerability indices and inspecting how these indices change as a function of steering effort, we can estimate the steerability of a model across various persona dimensions and directions. Our benchmark reveals that the steerability of many current models is limited -- due to both a skew in their baseline behavior and an asymmetry in their steerability across many persona dimensions. We release an implementation of our benchmark at https://github.com/IBM/prompt-steering.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 34 of its 47 references that are in the map. First posted on arXiv 19 Nov 2024.

Builds on

34 of 47 references.

References · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  4. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  5. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  6. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  7. 2023Whose Opinions Do Language Models Reflect?cited by 773
  8. 2023Larger language models do in-context learning differentlycited by 394
  9. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 207
  10. 2024Programming Refusal with Conditional Activation Steeringcited by 206
  11. 2024A Roadmap to Pluralistic Alignmentcited by 184
  12. 2023Fundamental Limitations of Alignment in Large Language Modelscited by 155
22 more
  1. 2023The Learnability of In-Context Learningcited by 127
  2. 2024Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaborationcited by 113
  3. 2023Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Dutiescited by 97
  4. 2024PersonaGym: Evaluating Persona Agents and LLMscited by 97
  5. 2023Guiding Large Language Models via Directional Stimulus Promptingcited by 95
  6. 2024Evaluating Large Language Model Biases in Persona-Steered Generationcited by 78
  7. 2023Word Embeddings Are Steers for Language Modelscited by 70
  8. 2024"My Answer is C": First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Modelscited by 69
  9. 2024Aligning LLMs with Individual Preferences via Interactioncited by 63
  10. 2023When Large Language Models contradict humans? Large Language Models' Sycophantic Behaviourcited by 62
  11. 2024DeAL: Decoding-time Alignment for Large Language Modelscited by 55
  12. 2023On the steerability of large language models toward data-driven personascited by 53
  13. 2023Steering Large Language Models for Machine Translation with Finetuning and In-Context Learningcited by 49
  14. 2024Steering Without Side Effects: Improving Post-Deployment Control of Language Modelscited by 46
  15. 2023What's the Magic Word? A Control Theory of LLM Promptingcited by 41
  16. 2024How Do Nonlinear Transformers Learn and Generalize in In-Context Learning?cited by 35
  17. 2024What are human values, and how do we align AI to them?cited by 32
  18. 2024Language Models in Dialogue: Conversational Maxims for Human-AI Interactionscited by 21
  19. 2024Exploring and steering the moral compass of Large Language Modelscited by 16
  20. 2024Propulsion: Steering LLM with Tiny Fine-Tuningcited by 14
  21. 2023Algorithmic Pluralism: A Structural Approach To Equal Opportunitycited by 13
  22. 2024CharED: Character-wise Ensemble Decoding for Large Language Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 16.

27 papers.

Sort:

By area:

  1. 2025AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agentscited by 22
  2. 2025A Course Correction in Steerability Evaluation: Revealing Miscalibration and Side Effects in LLMscited by 7
  3. 2025What's Producible May Not Be Reachable: Measuring the Steerability of Generative Modelscited by 6
  4. 2025Poor Alignment and Steerability of Large Language Models: Evidence from College Admission Essayscited by 6
  5. 2025What Can We Actually Steer? A Multi-Behavior Study of Activation Controlcited by 6
  6. 2025Quantifying the Impact of Structured Output Format on Large Language Models through Causal Inferencecited by 5
  7. 2025Security Steerability is All You Needcited by 4
  8. 2025Localizing Persona Representations in LLMscited by 3
  9. 2025An Auditable Agent Platform For Automated Molecular Optimisationcited by 2
  10. 2026AI Steerability 360: A Toolkit for Steering Large Language Modelscited by 2
  11. 2026FaithSteer-BENCH: A Deployment-Aligned Stress-Testing Benchmark for Inference-Time Steeringcited by 2
  12. 2026What Does the AI Doctor Value? Auditing Pluralism in the Clinical Ethics of Language Modelscited by 2
15 more
  1. 2025Inference-Time Intervention in Large Language Models for Reliable Requirement Verificationcited by 1
  2. 2025Music Recommendation with Large Language Models: Challenges, Opportunities, and Evaluationcited by 1
  3. 2026Left, Right, or Center? Evaluating LLM Framing in News Classification and Generationcited by 1
  4. 2026Predicting Biased Human Decision-Making with Large Language Models in Conversational Settingscited by 0
  5. 2026Emulating Aggregate Human Choice Behavior and Biases with GPT Conversational Agentscited by 0
  6. 2026Not All Subjectivity Is the Same! Defining Desiderata for the Evaluation of Subjectivity in NLPcited by 0
  7. 2026Toward Stable Value Alignment: Introducing Independent Modules for Consistent Value Guidancecited by 0
  8. 2026How Far Will They Go? Red-Teaming Online Influence with Large Language Modelscited by 0
  9. 2026On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performancecited by 0
  10. 2026Investigating Linguistic Steering: An Analysis of Adjectival Effects Across Large Language Model Architecturescited by 0
  11. 2026Auditing Alignment Controllability in LLMs via Political Axescited by 0
  12. 2026Controllable Affective Generation via Latent Vector Steeringcited by 0
  13. 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0
  14. 2026How Far Do Persona Effects Generalize in Language Models?cited by 0
  15. 2026MetaSteer: Context-Conditioned, nonlinear Steering via Attention-Projection Adaptationcited by 0