# When Do LLM Preferences Predict Downstream Behavior? arXiv: https://arxiv.org/abs/2602.18971 ยท HTML: https://mlsanity.com/paper/2602.18971/ ## Key facts - Cited by 7 papers in the map (data through 14 Oct 2026). - Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. - Builds on 24 of its 32 references that are in the map. - First posted on arXiv 21 Feb 2026. ## Placed in - LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/) ## Builds on (most cited first, 24 of 24) - [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022) - [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022) - [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023) - [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017) - [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019) - [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019) - [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023) - [Why Language Models Hallucinate](https://mlsanity.com/paper/2509.04664/) (2025) - [Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs](https://mlsanity.com/paper/2502.17424/) (2025) - [Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models](https://mlsanity.com/paper/2402.16786/) (2024) - [AI Sandbagging: Language Models can Strategically Underperform on Evaluations](https://mlsanity.com/paper/2406.07358/) (2024) - [Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals](https://mlsanity.com/paper/2210.01790/) (2022) - [Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs](https://mlsanity.com/paper/2502.08640/) (2025) - [Safety Cases: How to Justify the Safety of Advanced AI Systems](https://mlsanity.com/paper/2403.10462/) (2024) - [Uncovering Deceptive Tendencies in Language Models: A Simulated Company AI Assistant](https://mlsanity.com/paper/2405.01576/) (2024) - [Probing and Steering Evaluation Awareness of Language Models](https://mlsanity.com/paper/2507.01786/) (2025) - [Lessons from a Chimp: AI "Scheming" and the Quest for Ape Language](https://mlsanity.com/paper/2507.03409/) (2025) - [Will AI Tell Lies to Save Sick Children? Litmus-Testing AI Values Prioritization with AIRiskDilemmas](https://mlsanity.com/paper/2505.14633/) (2025) - [Generative Value Conflicts Reveal LLM Priorities](https://mlsanity.com/paper/2509.25369/) (2025) - [Stress-Testing Model Specs Reveals Character Differences among Language Models](https://mlsanity.com/paper/2510.07686/) (2025) - [Safety Cases: A Scalable Approach to Frontier AI Safety](https://mlsanity.com/paper/2503.04744/) (2025) - [Probing the Preferences of a Language Model: Integrating Verbal and Behavioral Tests of AI Welfare](https://mlsanity.com/paper/2509.07961/) (2025) - [Cognitive models can reveal interpretable value trade-offs in language models](https://mlsanity.com/paper/2506.20666/) (2025) - [Inertia in Moral and Value Judgments of Large Language Models](https://mlsanity.com/paper/2408.09049/) (2024) ## Built on by (most cited first, 7 of 7) - [The Consciousness Cluster: Emergent preferences of Models that Claim to be Conscious](https://mlsanity.com/paper/2604.13051/) (2026) - [Can Revealed Preferences Clarify LLM Alignment and Steering?](https://mlsanity.com/paper/2605.08556/) (2026) - [Persona Cartography: Charting Language Model Personality Traits in Weight Space](https://mlsanity.com/paper/2607.07916/) (2026) - [LLMs Contain Multitudes: How Deployment Context Reshapes Model-Level Preferences and Values](https://mlsanity.com/paper/2606.13944/) (2026) - [When Preferences Fail to Become Incentives: A Utility-Behavior Gap in Large Language Models](https://mlsanity.com/paper/2606.22974/) (2026) - [High-Stakes Decisions with Language Models: Insights from Emergency Triage](https://mlsanity.com/paper/2608.01361/) (2026) - [Strangers to Themselves: What Language Models Say About Themselves Is Generic](https://mlsanity.com/paper/2609.09899/) (2026) ## Abstract (arXiv) Preference-driven behavior in LLMs may be a necessary precondition for AI misalignment such as sandbagging: models cannot strategically pursue misaligned goals unless their behavior is influenced by their preferences. Yet prior work has typically prompted models explicitly to act in specific ways, leaving unclear whether observed behaviors reflect instruction-following capabilities vs underlying model preferences. Here we test whether this precondition for misalignment is present. Using entity preferences as a behavioral probe, we measure whether stated preferences predict downstream behavior in five frontier LLMs across three domains: donation advice, refusal behavior, and task performance. Conceptually replicating prior work, we first confirm that all five models show highly consistent preferences across two independent measurement methods. We then test behavioral consequences in a simulated user environment. We find that all five models give preference-aligned donation advice. All five models also show preference-correlated refusal patterns when asked to recommend donations, refusing more often for less-preferred entities. All preference-related behaviors that we observe here emerge without instructions to act on preferences. Results for task performance are mixed: on a question-answering benchmark (BoolQ), two models show small but significant accuracy differences favoring preferred entities; one model shows the opposite pattern; and two models show no significant relationship. On complex agentic tasks, we find no evidence of preference-driven performance differences. While LLMs have consistent preferences that reliably predict advice-giving behavior, these preferences do not consistently translate into downstream task performance.