# Steering Risk Preferences in Large Language Models by Aligning Behavioral and Neural Representations

arXiv: https://arxiv.org/abs/2505.11615 · HTML: https://mlsanity.com/paper/2505.11615/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability.
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 16 May 2025.

## Placed in

- Mechanistic interpretability / [Activation steering methods](https://mlsanity.com/mechinterp/activation-steering-methods/)

## Builds on (most cited first, 22 of 22)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [Getting aligned on representational alignment](https://mlsanity.com/paper/2310.13018/) (2023)
- [Inspecting and Editing Knowledge Representations in Language Models](https://mlsanity.com/paper/2304.00740/) (2023)
- [Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting](https://mlsanity.com/paper/1906.09531/) (2019)
- [Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs](https://mlsanity.com/paper/2502.08640/) (2025)
- [Large Language Models Assume People are More Rational than We Really are](https://mlsanity.com/paper/2406.17055/) (2024)
- [Large language models predict human sensory judgments across six modalities](https://mlsanity.com/paper/2302.01308/) (2023)
- [Incoherent Probability Judgments in Large Language Models](https://mlsanity.com/paper/2401.16646/) (2024)
- [Gibbs Sampling with People](https://mlsanity.com/paper/2008.02595/) (2020)
- [Eliciting the Priors of Large Language Models using Iterated In-Context Learning](https://mlsanity.com/paper/2406.01860/) (2024)
- [AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling](https://mlsanity.com/paper/2411.17284/) (2024)
- [Language Models Trained to do Arithmetic Predict Human Risky and Intertemporal Choice](https://mlsanity.com/paper/2405.19313/) (2024)
- [Recovering Mental Representations from Large Language Models with Markov Chain Monte Carlo](https://mlsanity.com/paper/2401.16657/) (2024)

## Built on by (most cited first, 6 of 6)

- [Can Revealed Preferences Clarify LLM Alignment and Steering?](https://mlsanity.com/paper/2605.08556/) (2026)
- [Auditing Asset-Specific Preferences in Financial Large Language Models: Evidence from Bitcoin Representations and Portfolio Allocation](https://mlsanity.com/paper/2606.02528/) (2026)
- [High-Stakes Decisions with Language Models: Insights from Emergency Triage](https://mlsanity.com/paper/2608.01361/) (2026)
- [Can Risk-taking AI-Assistants suitably represent entities](https://mlsanity.com/paper/2510.08114/) (2025)
- [Temporal Preference Concepts and their Functions in a Large Language Model](https://mlsanity.com/paper/2606.05194/) (2026)
- [The Internal Anatomy of Strategic Choice in Large Language Models](https://mlsanity.com/paper/2609.07478/) (2026)

## Abstract (arXiv)

Changing the behavior of large language models (LLMs) can be as straightforward as editing the Transformer's residual streams using appropriately constructed "steering vectors." These modifications to internal neural activations, a form of representation engineering, offer an effective and targeted means of influencing model behavior without retraining or fine-tuning the model. But how can such steering vectors be systematically identified? We propose a principled approach for uncovering steering vectors by aligning latent representations elicited through behavioral methods (specifically, Markov chain Monte Carlo with LLMs) with their neural counterparts. To evaluate this approach, we focus on extracting latent risk preferences from LLMs and steering their risk-related outputs using the aligned representations as steering vectors. We show that the resulting steering vectors successfully and reliably modulate LLM outputs in line with the targeted behavior.